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Prefatory  Note 


^ In  July  1976,  the  National  Security  Industrial  Association 
(NSIA)  sponsored  the  First  International  Learning  Technology 
Congress  in  Washington,  D.C.  As  part  of  the  Congress,  NSIA 
also  sponsored  the  Third  Triennial  Symposium  on  Cost  Effective 
Learning  Through  the  Application  of  Technology.  One  of  the 
six  sessions  which  made  up  the  Symposium  was  devoted  to  the 
topic,  “Instructional  Technology  in  the  Military.’V^his  session 
was  chaired  by  Dr.  Worth  Scanland  of  the  Center  for  Naval  Edu- 
cation and  Training,  Pensacola,  Florida.  \ 

Mr.  David  S.  Bushnell  of  HumRRO*s  Eastern  Division  r/as 
one  of  three  speakers  at  the  session  on  “Instructional  Technology 
in  the  Military.”  His  presentation  is  the  lead  item  in  this  Pro- 
fessional Paper.  It  was  supported  by  three  other  papers  written 
by  members  of  the  HumRRO  staff— papers  which,  although  not 
presented  at  the  Symposium,  will  appear  in  the  Symposium  pro- 
ceedings to  be  published  next  year.  These  three  “back-up” 
papers  are  also  included  in  this  Professional  Paper. 
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THE  IMPACT  OF  INSTRUCTIONAL  TECHNOLOGY  ON 
TRAINING  IN  THE  U.S.  ARMY 

• David  S.  Bushnell 


Almost  two  years  ago,  former  Secretary  of  Defense  James  Schlesinger  noted  that 
military  training  expenditures  exceeded  $6  billion  annually  and  involved  no  less  than  1/6 
of  all  military  personnel  at  any  given  time  (1).  Recognizing  the  Defense  Department’s 
heavy  investment  in  manpower,  Schlesinger  requested  that  a Task  Force  on  Training  Tech- 
nology be  convened  and  given  the  responsibility  of  evaluating  the  effectiveness  of  DOD 
training.  Their  objective:  to  recommend  ways  of  reducing  costs  and  increasing  effective- 
ness of  DOD  training. 

Working  over  a 12-month  interval,  from  July  1974  to  June  1975,  the  Task  Force 
concluded  that  too  few  dollars  were  being  expended  on  R&D  training  technology  (0,05% 
of  the  total  DOD  research  outlay)  and  that,  even  with  this  relatively  small  investment, 
considerable  progress  had  been  made  in  promoting  cost-effective  training  (2).  Furthermore, 
they  observed  that  “the  Services  have  pioneered  (a)  in  the  use  of  complex  simulators  to 
train  personnel  to  operate  and  maintain  major  weapons  systems,  (b)  in  self-paced  person- 
alized methods  of  instruction,  (c)  in  performance-oriented  training,  and  (d)  in  managing 
the  tr^ning  of  very  large  numbers  of  individuals”  (3).  But,  they  observe,  insufficient 
attention  is  being  ^ven  to  collective  training,  that  is,  the  training  of  crews,  groups,  teams, 
and  units  at  a decentralized  level.  These  same  sentiments  were  reiterated  four  months 
later  when,  at  the  8th  NT  EC  and  Industry  Conference  in  Orlando,  Florida,  Army  Major 
General  Paul  Gorman,  Deputy  Chief  of  Staff  for  Training,  told  attendees  that  “ ...  in 
this  year  of  1975,  the  Captains  of  the  United  States  Army  and  the  Marine  Corps  face 
challenges  beyond  any  that  gents  of  their  rank  have  ever  faced  in  combat  in  the  history 
of  the  United  States.”  He  then  went  on  to  say  that  both  the  Army’s  officers  aiid  its 
enlisted  men  will  be  required  to  make  increasingly  critical  decisions  within  shorter  periods 
of  time  (4).  The  growing  complexity  of  our  weapon  systems  demands  sophisticated  training 
programs  where  simulated  engt^ement  exercises  are  used  to  polish  decision-making  skills. 

The  effective  opieration  and  maintenance  of  today’s  weapon  systems  requires  increasingly 
able  and  experienced  persoiuiel.  How  the  Army  has  responded  to  these  demands  for 
improved  training  programs  since  World  War  II  is  a story  well  worth  the  telling. 

When  Dr.  Scanland  wrote  me  several  months  ago  to  present  a paper  on  this  topic  at 
this  first  International  Learning  Technology  Congress,  I was  delighted  to  do  so.  In  his 
own  persuasive  manner.  Dr.  Scanland  asked  if  I would  like  to  address  myself  to  the  advances 
that  my  organization.  Human  Resources  Research  Organization  (HumRRO),  has  contributed 
to  Army  training  over  the  last  25  years  through  research  and  development.  As  if  this 
weren’t  enough,  he  then  sweetened  the  pot  by  inviting  me  to,  in  tvim,  invite  three  or  four 
of  my  o^  colleagues  to  prepare  back-up  papers  further  detaUing  HumRRO ’s  close  work- 
ing relationship  with  the  Army  and  contributions  in  specific  areas.  To  top  it  off,  he 
offered  to  publish  all  four  papers  as  part  of  the  proceedings  of  this  Conference.  Thus, 
much  of  what  follows  draws  heavily  upon  the  contributions  of  my  colleagues,  with  the 
caveat  that  some  of  the  rasher  generalizations  which  I make  lator  on  in  my  talk  are  uniquely 
my  own  and  do  not  necessarily  reflect  the  views  of  HumRRO  or  my  associates. 

For  a more  complete  and  carefully  reasoned  statement  of  specific  HumRRO  experi- 
ences touched  on  briefly  here,  I refer  you  to  Mr.  Lavisky’s  historical  review  of  Army 
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research  on  training.  Dr.  McFann’s  and  Dr.  Taylor’s  discussion  of  development  of 
performance-oriented  training,  and  Dr.  Caro’s  review  of  res^ch  on  the  use  cf  simula- 
tors in  pilot  training. 

Since  1951,  HumRRO  scientists  have  been  discovering,  developing,  and  applying 
human  factors  and  behavioral  science  principles  to  the  improvement  of  training  and 
individual  performance  of  Army  personnel.  Perhaps  the  best  way  for  me  to  give  you  a 
“feel”  for  the  impact  that  our  research  and  development  program  has  had  on  the  Army, 
would  be  for  me  to  briefly  describe  Army  developments  in  (1)  basic  combat  training, 

(2)  advanced  training,  (3)  support  training,  and  (4)  flight  training  over  the  last  35  years. 

I will  conclude  my  remarks  by  attempting  to  identify  for  you  some  of  the  seminal  forces 
which  have  helped  to  bring  about  these  needed  improvements  in  training  technology. 

Since  the  outbreak  of  World  War  II,  I think  you  will  agree  that  dramatic  progress  has 
been  made. 

Mr.  Lavisky  in  his  companion  paper,  “Army  Research  on  Training;  How  it  All  Began,” 
observes  that  before  World  War  II,  Army  training  was  decentralized,  highly  personalized, 
and  unstandardized.  Because  of  the  small  number  of  enlistees  each  year,  recruits  were 
trained  within  the  regiment  which  they  joined  and  were  likely  to  remain  with  that  regiment 
imtil  discharge.  Performance  standards  were  set  by  the  regimental  commander.  Let  me 
underscore  two  key  terms  I just  used— “decentralized”  and  “highly  personalized.”  Those 
are  terms  that  will  pop  up  aeain  in  our  discussion.  I think  it  is  a reasonably  safe  observa- 
tion that  before  the  onset  of  our  involvement  in  World  War  II,  both  the  Army  and  Navy, 
but  particularly  the  Army,  were  faced  with  limited  dollars  for  training  with  the  apprentice- 
ship model  serving  as  the  principle  guide.  This  status  reflected  the  failure  of  the  military 
between  World  War  I and  II  to  involve  American  psychologists  and  other  behavioral 
scientists  in  military  problems,  with  the  possible  exception  of  the  improvement  of  the  per- 
sonnel selection  system  (5). 

In  1939,  the  Army  was  forced  tc  change  its  approach  to  training  in  order  to  accommo- 
date the  masses  of  soldiers  brought  into  the  Service  through  mobilization.  Between  1939 
and  1941,  for  example,  three  incomplete  Infantry  Divisions  grew  to  thirty  Infantry  Divi- 
sions backed  up  by  six  Armored  Divisions.  The  training  model  employed  was  not  developed 
through  research,  but  essentially  followed  the  traditional  lecture-oriented,  information- 
centered,  time-constrained  approach  found  in  most  colleges  and  public  schools  at  that  time. 
The  “pointer /podixim/poop”  approach  to  Army  training  was  the  product  of  three  factors: 

1.  The  need  to  train  a large  number  of  officers,  to  be  used  subsequently  in  the 
training  of  enlisted  personnel,  resulted  in  a heavy  reliance  on  the  traditional 
lecture  method.  Such  a model  was  adopted  by  default.  No  other  learning 
approach  was  around  to  compete  with  it. 

2.  The  rapid  build-up  of  forces  put  pressure  on  military  leaders  to  standardize 
subject-matter  content  through  the  adoption  of  field  manuals  as  the  focal 
point  of  training.  Since  such  manuals  were  organized  around  subject  matter, 
the  training  itself  was  necessarily  focused  on  subject  matter.  Visual  aids  were 
extensively  used,  putting  even  greater  emphasis  on  passive  listening  rather  than 
active  involvement  in  the  learning  process. 

3.  The  time  avmlable  for  training  and  the  sequence  of  the  subject  matter  were 
also  standardized.  Before  then,  each  company,  troop,  or  battery  had  its  own 
training  schedule  determined  largely  by  the  Unit  Commander’s  evaluation 

of  needs  and  time  constraints.  Eventually,  however,  greater  efficiency  and 
better  coordination  was  realized  through  standardizing  not  only  the  order  of 
subjects  to  be  taught,  but  the  amount  of  time  devoted  to  each  subject 
as  well  (6). 

It  would  be  inaccurate,  however,  to  characterize  basic  training  as  totally  lacking  in 
the  improvement  of  performance  skills.  Target  shooting  and  close-order  drill  were  common 
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events.  Extensive  use  of  Army  maneuvers  with  simulated  battle  sounds  and  conditions 
was  used  to  help  condition  Army  units  to  combat.  Few  among  us  will  forget  the  day  that 
we  were  required  to  crawl  on  our  bellies  with  a full  pack  under  a blanket  of  live  machine- 
gun  bullets.  We  learned  to  keep  our  heads  down. 

Following  World  War  II,  the  high  cost  of  ammimition  together  with  mounting 
evidence  that  basic  training  as  constituted  during  the  war  years  had  a negative  effect  on 
recruit  attitudes  toward  the  Army,  resulted  in  Army  Training  Centers  adopting  a new  set 
of  procedures  directed  at  increasing  the  trainees’  identification  with  his  unit  and  with  the 
Army.  It  was  about  this  time  that  the  Human  Resources  Research  Office  (HumRRO) 
h^gan  to  contribute  to  the  improvement  of  basic  training  through  the  application  of 
behavioral  and  social  science  concepts. 

After  the  War,  the  Army’s  Adjutant  General’s  Office,  with  its  primary  responsibility 
for  the  improvement  of  personnel  selection  and  placement,  continued  to  focus  much  of 
its  attention  on  testing.  In  1947,  a small  cadre  of  personnel  researchers  turned  their 
attention  to  measuring  the  results  of  basic  training  and  of  career  schooling.  By  1950, 
the  Army  began  to  recognize  that  it  was  not  keeping  pace  with  either  the  Navy  or  the 
Air  Force  in  sponsoring  or  benefiting  from  research  on  the  improvement  of  human  per- 
formance. A staff  study  memo  dated  June  22,  1950  states:  “The  annual  budget  alloca- 
tions for  research  in  human  resources  in  the  Department  of  the  Army  has  been  1%  or 
less  of  the  total  research  and  development  budget  for  the  entire  Department  of  the  Army.” 
The  upshot  of  this  and  other  staff  studies  resulted  in  the  Army  contracting  with  the 
George  Washington  University  for  the  establishment  of  HumRRO.  Its  goals,  as  shaped  by 
Dr.  Harry  Harlow,  at  that  time  the  Army’s  Chief  Psychologist,  were  threefold: 

1.  To  conduct  research  in  the  areas  of  training  methods,  motivation  and 
morale,  and  psychological  warfare  techniques. 

2.  To  set  up  a civilian  research  staff  at  a central  location  empowered  to  grant 
and  monitor  contracts  to  educational,  buaness,  and  industrial  organizations 
as  well  as  provide  technical  supervision  of  research  conducted  at  military 
installations. 

3.  To  establish  in-service  research  units  at  appropriate  military  installations  who 
were,  in  turn,  directed  to  work  with  HumRRO’s  staff  on  a collaborative 
basis. 

During  the  first  four  years  of  HumRRO  effort,  we  learned  that  the  training  context 
provided  the  most  productive  area  of  focus  in  attacking  morale,  motivation,  leadership, 
and  training  problems.  Research  goals  were  specified  in  an  Annual  Work  Program  ini- 
tially agreed  upon  by  the  Army  and  HumRRO.  Organizationally,  three  research  divi- 
sions, the  Director’s  office,  and  supporting  services  were  initiaUy  located  on  the 
University’s  campus  and  later  in  Alexandria,  Vh^nia.  By  1958,  five  field  divisions  had  been 
formed  and  co-located  with  military  research  imits  at  major  Army  installations  throughout 
the  country.  Almost  from  the  beginning,  the  HmnRRO  research  staff  was  augmented  by 
Army  personnel  located  at  the  field  division  sites  or  detailed  for  work  at  corporate  head- 
quarters in  Alexandria,  Vii^ia. 

Basic  Combat  Training  (BCT) 

Among  the  first  series  of  studies  conducted  by  HumRRO  was  an  evaluation  of  the 
impact  of  basic  training  on  Army  recruits.  (See,  for  example,  references  7 and  8.)  One 
of  the  more  significant  products  of  that  research,  aimed  at  increasing  the  recruit’s  identi- 
fication with  bis  unit  and  with  the  Army,  was  the  cstablislunent  of  a five-week  training 
program  for  Army  Drill  Sergeants.  A four-man  HumRRO  team  worked  for  several  months 
wth  Army  tnuning  personnel  to  develop  a training  program  which  would  help  the  Drill 
Sergeant  learn  more  about  motivating  recruits,  developing  trainee  skills,  and  building 
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esprit  de  corps.  Please  note  that  HumRRO  personnel  did  not  design  such  a training 
program  in  isolation,  but  spent  several  months  in  close  collaboration  with  Army  train- 
ing personnel.  Subsequently,  Army  Drill  Sergeant  schools  were  established  at  a number 
of  Army  installations. 

Another  major  change  in  recruit  training  came  about  in  1957  when  the  Army  adopted 
a rifleman  training  procedure  (for  use  during  both  day  and  night)  which  put  stress  upon 
first  studying  and  then  simulating  the  combat  rifleman’s  job.  Before  that,  marksmanship 
skills  were  developed  by  having  a recruit  fire  at  bull’s-eye  targets  at  known  distances  just 
as  he  might  in  competitive  marksmanship  matches.  Under  the  new  procedure  (9),  the 
trainee  was  taught  to  spot  and  estimate  the  distance  of  a pop-up  silhouette  target  (dubbed 
“Punchy  Pete”)  which  dropped  when  it  was  hit.  Night-firing  marksmanship  programs, 
based  on  the  concept  of  aligning  the  rifle  without  using  its  sights,  resulted  in  a 60%  to 
210%  increase  in  accuracy  over  previous  training  methods.  Here  again,  HumRRO  researchers 
worked  on-site  at  all  of  the  major  Army  training  locations,  including  Forts  Knox,  Benning, 
Rucker,  Bliss,  Ord,  and  later  on,  at  Fort  Hood.  Studies  were  also  conducted  in  the  field 
at  a variety  of  Army  operational  sites,  including  Europe,  Korea,  and  Viet  Nam.  Close 
collaboration  with  military  personnel  helped  to  ensure  adoption  and  implementation  of 
recommendations  derived  from  the  research. 

Most  of  the  BCT  studies  (and  subsequent  efforts)  had  as  their  origin  a specific  opera- 
tional problem.  Every  effort  was  then  made  to  generalize  the  finding  to  other  comparable 
rituations.  Thus,  the  value  and  utility  of  HumRRO  work  tended  to  follow  an  experimental 
curve  of  usefulness  over  time.  The  degree  of  success  achieved  in  solving  problems  was 
thought  to  be  a function  of  timeliness,  changes  in  Army  priorities  (compatibility  with 
established  practices),  skill  in  communicating  results  to  key  decision  makers,  and  charac- 
teristics of  the  innovation  itself,  e.g.,  its  obvious  advantage  over  older  practices  in  terms 
of  efficiency  (10).  Our  close  proximity  to  and  collaboration  with  Army  personnel  helped 
gain  acceptance  for  our  findings  and  recommendations  as  well  as  establish  the  credibility 
of  the  HumRRO  effort. 

One  dramatic  illustration  of  this  occurred  in  1966  when  the  Army  adopted  a service- 
wide  regulation  (CONARC  Reg  350-100-1)  which  was  used  to  guide  the  development  of 
all  training  programs  between  1966  and  1971.  This  “systems  engineering”  approach 
required  that  precise  behavioral  objectives  for  training  be  developed  through  a careful  job 
analysis  and  that  the  specific  subject  matter  selected  for  the  training  program  be  included 
on  the  basis  of  its  relevance  to  the  tasks  to  be  performed.  Criterion-referenced  tests  designed 
to  assess  individual  mastery  of  various  performance  objectives  were  also  introduced  (11). 

Thus,  up  to  1966,  HumRRO’s  integrated  research  and  development  program  not  only  focused 
on  improving  basic  training,  but  it  was  equally  concerned  with  job  analysis  and  evaluation 
procedures,  the  improvement  of  instructional  methods,  and  developing  a better  understanding 
of  the  learring  styles  and  characteristics  of  trainees.  Testing  and  evaluation  procediures 
developed  at  the  same  time  helped  to  ensure  that  the  basic  recruit,  upon  completion  of 
basic  training,  could  perform  the  necessary  skills  and  could  demonstrate  that  he  had  the 
knowledge  required  of  a foot  soldier. 

Advanced  Training 

Even  with  these  advances,  much  remained  to  be  done  in  the  areas  of  Advanced  Indi- 
vidual Training  (AIT)  and  Support  Training  (ST).  The  pattern  adopted  during  World  War  II 
of  lecture-centered,  standardized  courses  oriented  to  group  instruction  persisted  in  the  class- 
room. Even  in  the  periodic  renewal  of  manual  skills,  such  as  rifle  firing,  little  attention  was 
^ven  to  more  optimal  schedules  of  reinforcement  designed  to  maximize  skill  retention. 
Fortunately,  outside  the  classroom,  a number  of  exceptions  occurred. 

After  completion  of  BCT,  a recruit  was  often  assigned  to  rifle  squad  training  where 
techniques  of  fire  arid  patrolling  skills  were  taught,  based  on  HumRRO  analyses  of  what 
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a soldier  must  know  and  do  in  actual  combat  (12).  In  addition,  eight  fundamental  night- 
operation  skills  were  taught.  These  too,  were  derived  from  field  observations  by  HumRRO 
scientists  (13). 

Had  the  BCT  graduate  been  assigned  to  other  AIT  programs,  such  as  Armor  or  Air 
Defense,  he  would  have  been  presented  with  on-the-job  aids  in  a functional  context.  Here 
the  material  to  be  learned  was  broken  down  into  a step-by-step  procedure  with  specific 
instructional  techniques  for  use  by  recently  graduated  peer  instructors.  Air  Defense  missile 
gunners  were  taught  to  recognize  enemy  aircraft  through  a procedure  developed  by 
HumRRO  personnel  (14).  The  Combat  Development  Command  recommended  that  this 
prototype  program  be  adopted  for  training  all  forward -area  air  defense  gunners. 

Support  Training 

Recruits  graduating  from  basic  training  were  also  assigned  to  non-combat  roles  in  what 
were  called  “support”  positions.  Radio  operators,  medical  corpsmen,  supplymen,  and  cooks 
all  benefited  from  research  on  how  to  formulate  training  objectives,  select  course  content, 
and  alternative  instructional  methods  and  procedures  (aimed  at  increasing  training  efficiency 
and  reducing  attrition).  This  “systems  engineering  approach”  resulted  in  a 10%  to  15% 
reduction  in  training  time  while  maintaining  or  enhancing  the  performance  of  graduates. 

In  1966,  a major  new  recruitment  policy  was  adopted  by  all  the  Services.  They  agreed 
to  dramatically  increase  the  number  of  lower  mental  ability  recruits  accepted  into  military 
service.  HumRRO  was  asked  to  undertake  a series  of  studies  to  determine  what  effect 
this  policy  would  have  on  the  training  and  utilization  of  personnel  in  the  Army.  It  was 
feared  that  the  Army’s  continued  reliance  upon  passive  classroom  lecture  and  paper-and- 
pencil  (normative)  tests  would  prove  to  be  ineffective  with  lower  mental  ability  personnel. 

Shortly  thereafter,  collie  deferrals  were  abandoned.  The  resulting  heterogeneity  of 
recruits  demanded  an  even  more  flexible  form  of  instruction.  Between  1968sand  1972 
the  HumRRO  research  and  development  effort  focused  on  devising  instructional  approaches 
which  would  accommodate  widely  varing  aptitude  levels.  As  an  example  of  how  this  series 
of  new  studies  was  organized,  let  me  describe  in  somewhat  more  detail  a recent  systems 
engineering  effort  of  eight  Combat  Arms  Military  Occupational  Speciality  Codes  (MOSs). 

The  Combat  Arms  Training  Board  (CATB)  of  the  Army  had  discovered  that  those 
responsible  for  training  operational  units  needed  assistance  ip  increasing  the  proficiency 
level  of  their  non-commissioned  officers.  Trainers  at  the  unit  level  had  neither  the  meth- 
odology nor  the  material  to  conduct  adequate  training.  The  combat  arms  soldiers  were 
also  complaining  tiiat  they  had  little  chance  to  develop  their  skills  beyond  the  rudimentary 
level.  The  first  step  taken  in  solving  this  problem  was  to  attempt  a simultaneous  systems 
engineering  of  eight  Combat  Arms  MOSs.  We  were  able  to  take  advantage  of  the  common- 
alities among  tasks  that  existed  between  various  duty  positions.  In  this  way  the  training 
could  be  made  more  efficiait  if  common  tasks  were  identified  and  training  materials 
developed  once  rather  than  several  times  fay  different  combat  arms  trainers. 

In  this  clustering  study,  93  different  duty  positions  were  identified  and  task  inventories 
developed  on  each.  A classification  system  was  used  to  permit  comparison  of  the  material 
from  each  position.  The  tasks  were  identified  by  category,,  such  as  first  aid,  land  naviga- 
tion, and  tactics.  Within  each  category  tasks  were  broken  out  into  three  levels;  (1)  funda- 
mental-Hhose  tasks  expected  to  be  common  to  all  soldiers  in  the  Army;  (2)  branch— those 
tasks  expected  to  be  common  to  duty  positions  in  a given  branch;  and  (3)  MOS— those 
tasks  associated  with  specific  duty  positions.  Task  descriptions  were  validated  by  three 
different  groups;  job  incumbents,  senior  NCOs,  and  officers.  Job  task  data  (e.g.,  job 
conditions,  knowledge  and  skill  requirements)  were  developed  for  all  tasks  that  were  per- 
formed by  60%  or  more  of  the  job  incumbents.  That  information  was  then  used  to  develop 
training  objectives,  training  materials,  reference  manuals,  evaluation  criteria,  and  MOS  test 
questions.  The  information  thus  developed  proved  to  be  an  important  input  for  curriculum 
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■ planners,  training  administrators,  and  training  developers  at  each  of  the  Combat  Arms 
Schools.  This  required  an  immense  amount  of  coordination  between  the  Army,  three 
of  HumRRO’s  research  divisions,  and  CATB  (15). 

Emei^ng  out  of  the  mass  of  recent  research  on  the  training  process  are  six  principles 
which  deserve  to  be  recapitulated  here.  They  are; 

1.  Performance-based  Instruction.  Students  learn  the  skills  necessary  for  job 
performance.  The  emphasis  is  on  active  skill  practice— “doing”  rather 
than  “passive  absorption  of  information.” 

2.  Task  Mastery.  Every  student  is  required  to  reach  a particular  standard  of 
performance  in  each  skill.  Assessment  is  on  a “go/no-go”  basis.  The 
student  who  does  not  reach  the  criterion  level  of  performance  receives 
additional  practice  until  he  does  reach  it. 

3.  Functional  Context.  The  student  learns  in  a job-relevant  situation.  Theo- 
retical or  technical  material  is  presented  only  when  it  is  needed  in  learning 
to  perform  a skill. 

4.  Individualization . For  various  reasons  people  learn  at  different  rates.  To 
the  extent  possible,  the  trainee  is  permitted  to  learn  a skill  at  his  own  rate. 

5.  Feedback.  If  the  instructor  and  curriculum  developer  know  a good  ideal 
about  the  student  to  be  taught,  instructional  methods  can  be  ror>dified  to 
be  more  effective.  If  the  student  knows  about  his  own  skill  acquisition, 
he  is  motivated  to  correct  enors  of  inadequate  performance. 

6.  Quality  Control.  To  ascertain  that  the  training  system  is  functioning  properly, 
student  performance  must  be  systematically  assessed  at  various  times  during 
and  at  the  end  of  training. 


Flight  Training 

Before  I bring  my  remarks  to  a close,  I would  like  to  comment  on  one  other  area  of 
HumRRO  research;  the  training  of  helicopter  pilots.  As  everyone  knows,  aviation  train- 
ing is  costly.  The  high  cost  of  fuel  combined  with  the  increasing  value  and  complexity  of 
Army  aircraft  led  HumRRO  to  propose  to  the  Army  that  it  attempt  to  improve  its  evalua- 
tion of  pilot  psrformance  and  to  employ  less  costly  flight  trainers  in  teaching  maneuvers. 

As  a result  of  this  research,  the  Army  helicopter  trainee  gets  his  first  taste  of  rotary -wing 
flight  in  a helicopter  training  device.  Men  trained  on  this  device  experienced  signifi- 
cantly less  attrition  during  the  subsequent  flight  training— only  10%  attrition  compared  with 
30%  attrition  under  previous  procedures— and  they  made  better  grades  and  soloed  earlier  (16). 

Paul  Caro’s  excellent  review  of  research  on  the  role  of  simulators  in  pilot  training 
explores  those  factors  which  influence  the  transfer  of  simulated  training  to  operational 
aircraft.  Six  factors  were  shown  as  having  significant  impact  on  simulator  training  effec- 
tiveness. Simulator  design,  visual  fidelity,  motion  fidelity,  handling  characteristics,  etc. 
were  judged  from  the  vantage  point  of  their  influence  upon  training  effectiveness.  Other 
variables  (such  as  the  sequence  of  instruction  employed,  individual  vs.  group  pacing, 
training  to  specified  criterion  levels,  and  length  of  exposure  to  training  on  a simulator) 
need  investigation  if  the  proper  combination  of  hardware,  training  program,  and  personnel 
is  to  be  ascertained.  Caro  concluded  that  while  simulator  training  artisans  can  sometimes 
produce  spectacular  results,  there  are  too  few  such  individuals  to  develop  and  continuously 
update  all  of  the  simulator  training  programs  required  by  military  and  civilian  pilot  train- 
ing establishments  and  operational  units.  He  concludes  his  paper  with  a plea  for  more 
case  study  reports  of  simulator  training  applications  so  that  conceptual  models  might  be 
developed  for  future  applications  and  research.  Gathering  data  on  existing  simulators  and 
training  programs  would,  he  argues,  provide  a broad  data  base  for  judging  the  effectiveness 
of  simulator  training.  Through  this  approach,  extensive  savings  might  be  realized  through 
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designing  simulators  to  meet  training  specifications.  One  such  device,  wh.ose  commercial 
procurement  costs  would  have  been  about  $75,000,  was  fabricated  by  the  Army  to 
HumRRO  specifications  at  a cost  of  less  than  $5,000. 

Summary  and  Conclusions 

What  began  as  a decentralized,  somewhat  individualized  and  unstandardized  approach 
to  pre-World  War  II  training  has  come  full  circle.  While  it  would  be  misleading  to 
characterize  today’s  training  principles  as  identical  with  those  followed  during  the  period 
between  1918-1939,  they  are  parallel  in  the  sense  that  greater  emphasis  is  now  being 
placed  on  decentralized  decision  making,  expanded  use  of  self-paced  and  individualized 
instruction,  and  greater  flexibility  in  training  approach.  What  has  emerged  from  a 'us- 
tained  period  of  research  and  development  is  a more  pragmatic,  sophisticated  r^pproi  h to 
the  orderly  development  of  Army  training.  Starting  with  a careful  and  detailed  analysis 
of  a particular  job  function  and  progressing  step  by  step  through  the  specification  of 
needed  knowledges  and  skills  to  the  determination  of  training  objectives  and  the  organization 
and  evaluation  of  the  training  program,  HumRRO  has  helped  to  create  a generalizable  pro- 
cedure that  has  proved  to  be  of  considerable  value  as  new  training  tasks  are  undertaken. 

In  closing,  it  should  be  recognized  that  the  procedures  just  outlined  are  only  as  good 
as  the  learning  theory  upon  which  they  are  based.  The  interaction  between  theory  and 
empirical  observation,  coupled  with  the  involvement  oi  military  personnel  in  priority 
setting  and  the  implementation  of  results,  has  resulted  in  an  unusually  productive  and 
beneficial  relationship. 
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ARMY  RESEARCH  ON  TRAINING:  HOW  iT  ALL  BEGAN 

Saul  Lavisky 


The  Seeds:  Selection  and  Classification  Procedures 

At  the  oiitset  of  World  War  II,  the  U.S.  Army  already  had  a 166-year  history  of 
training  its  soldiers,  and  had  never  lost  a war.  By  contrast,  educational  research  and 
psychological  research  on  education  had  not,  by  1941,  achieved  much  success  in  improv- 
ing the  nation  *s  educational  practice.  How  did  it  come  to  pass  that  the  Army  txumed  to 
psychology  for  help  in  the  immensely  practical  task  of  training  its  soldiers? 

It  would  be  as  wrong  to  suggest  that  Army  interest  in,  and  use  of,  scientific  training 
research  resulted  from  some  grand,  preconceived  scheme  as  it  would  be  to  suggest  that 
the  Army  arrived  at  its  current  advanced  stage  of  training  technology  by  a series  of 
fortuitous  accidents. 

The  seeds  of  psychological  assistance  in  Army  operations  were  actually  sown  in 
World  War  I,  when  a special  committee  of  the  American  Psychological  Association  began 
exploring  methods  and  procedures  for  testing  and  classifying  new  recruits.  (1)  The 
psychological  profession  was  a small  and  relati^rely  new  one  at  that  time;  the  American 
Psychological  Association  had  only  336  members  in  1917.  But  this  group  quickly  estab- 
lished its  utility  to  the  Army  by  adapting  some  “exploratory  IQ  tects”  into  the  Army 
Alpha  group  test  for  classifying  literates  and  the  Army  Beta  for  classifying  illiterates. 

It  is  not  an  overstatement  to  describe  these  tests  as  “landmarks”  in  the  mental  measure- 
ments movement— landmarks  paralleling  the  individual  intelligence  test  devised  by  Binet 
early  in  the  twentieth  century. 

Much  of  the  other  work  that  psychologists  performed  for  the  Army  during  World 
War  is  now  noted  only  m specialized  histories  Yet  a list  of  those  psychologists  includes 
the  “giants”  of  the  profession:  R3I.  Yerkes  (who  directed  the  program);  R.S.  Woodworth 
(who  developed  tests  to  predict  courage  unda  stress);  Edward  L.  Thorndike  (who  devised 
tests  to  identify  potenthaiy  successful  aviators);  and  J.R.  Angell,  Louis  M.  Terman,  and 
John  B.  Watson  (who  initiated  development  of  classification  procures.)  Also  included 
■were  W.S.  Hunter,  L.L.  Thurstone,  G.M.  Whipple,  Edvrin  G.  Boring,  Horace  B.  English, 
and  Arthm:  S.  Otis.  (2) 

When  World  War  I ended,  these  men  returned  to  their  civilian  pursuits,  as  did  most 
of  the  1,726,966  soldiers  who  had  been  “psychologically  examined,”  in  groups  and 
individually,  on  tests  they  had  developed. 

In  the  years  between  World  War  I and  World  War  II,  American  psychology  advanced 
on  a number  of  non-military  fronts,  including  the  psychological  measurement  of  human 
aptitudes  and  abilities  and  educational  psychology. 

When  General  George  C.  Marshall  became  Army  Chief  of  Staff  in  1939,  he  estab- 
lished a Personnel  Testing  Section  in  hb  Adjutant  General’s  Office  and  set  that  group 
to  work  developing  the  first  edition  of  the  Army  General  Classification  Test  (AGCT). 

The  test  was  first  used  in  1940.  Fortunately,  it  was  available  when  Congress  pissed  the 
Selective  Service  Act  and  the  “draft”  began  to  present  new  men  by  Ihe  tens  of|thousands 
for  Army  service.  As  the  Army  moved  to  a wartime  footing,  the  Personnel  Te^g  Section 
was  enlarged  into  a Personnel  Research  Section  which  developed  a whole  varietf  of 


election  and  other  types  of  “predictive”  tests  in  support  of  the  Army  personnel  system. 
These  included  achievement  and  aptitude  tests,  selection  programs,  and  tests  of  all  kinds 
of  abilities.  Even  attitudinal  research  was  conducted  by  the  Army’s  psychologists  and 
sociologists.  (3) 

We  need  to  remember  that,  at  this  point  in  American  military  history,  there  was  no 
separate  air  arm.  The  Army  had  its  earthbound  soldiers  in  the  Aimy  Ground  Forces  and 
Amy  Service  Forces,  and  its  airmen  in  the  Army  Air  Forces.  All  were  under  the  com- 
mand of  a single  Chief  of  Staff,  General  George  C.  Marshall. 

General  Marshall  is  known  to  have  been  extremely  concerned  not  to  repeat  the  train- 
ing mistakes  of  World  War  I,  which  had  seen  so  many  ill-prepared  soldiers  shipped  from 
the  U.S.  to  the  trenches  in  France.  But,  from  the  time  he  became  Chief  of  Staff  in  1939 
xmtil  just  before  the  attack  on  Pearl  Harbor,  he  had  to  spend  most  of  his  time  and 
energy  enlarging  the  Army,  housing,  and  equipping  it.  Even  today,  35  years  after  the 
fact,  his  role  in  the  passage  of  the  Selective  Service  Act,  in  its  extension  (less  than  a year 
later),  and  in  the  call-up  and  retention  of  the  National  Guard,  makes  exciting  reading.  (4- 

Army  Training 

Before  World  War  II,  Army  training  had  been  decentralized,  highly  personalized,  and 
extremely  unstandardized.  This  kind  of  training  “system”  was  possible  because  of  the 
relatively  small  number  of  new  enlistees  each  year.  The  recruit  learned  basic  soldiering 
skills  as  well  as  the  rudiments  of  his  job  within  the. regiment  in  which  he  would  ser/e— a 
military  version  of  apprenticeship  training. 

With  World  War  II  on  the  horizon,  it  was  necessary  in  1939  for  the  Army  to  change 
its  approach  to  training  in  order  to  accommodate  the  masses  of  soldiers  it  had  to  train— 
hastily— for  both  the  mobilization  and  casualty-replacement  stages  of  the  war.  Training 
was  centralized  in  Replacement  Training  Centers  (one  or  more  for  each  branch),  deper- 
sonalized (because  of  the  large  numbers  of  recruits  involved  and  the  constant  turnover  in 
trainers),  and  relatively  standardized  (on  paper,  if  not  in  fact). 

The  Ground  Army  was  nearly  overwhelm^  ty  the  job.  Between  1939  and  1941, 
for  example,  it  grew  from  three  incomplete  Infantry  Divisions  to  30  Infantry  Divisions, 
complete  in  number  and  improving  in  quality,  bae'sed  up  by  six  Armored  Divisions. 

The  fledgling  Army  Air  Corps  was  producing  30,000  pilots  per  year,  and  was  planning 
for  37,000  new  pilots  in  1942.  By  the  time  the  build-up  was  complete,  the  Army  had 
100  divisions.  (5) 

Although  the  Army  Ground  Forces— the  major  command  responsible  for  training 
ground  combat  troops— did  not  sponsor  or  support  any  formal  research  to  improve  its 
instructional  programs,  it  did  attempt  to  use  the  be,?t  available  “technology.”  Some  of 
these  attempts  were  successful.  Some  were  considerably  less  so. 

The  Armored  School  at  Fort  Knox,  Kentucky,  representing  one  of  the  Army’s  newc-st 
branches,  prepared  an  elaborate  program  for  its  inco.ming  instructors.  The  Commandant 
turned  first  for  assistance  to  civilian  industrial  educar.ors.  The  program  they  designed 
had  new  instructors  meeting  an  hour  a day  for  three  months.  B\it  the  subject-matter  was 
too  arcane  for  the  audience— psychology,  statistics,  job  analysis,  learning  theory— and  not 
a reference  to  Field  Manual  21-5  on  “Methods  of  Instruction,”  with  which  aU  the  instruc- 
tors were  reasonably  well  acquainted. 

In  less  than  a year,  the  Commandant  fired  his  civilian  “experts”  and  brought  in  a 
Reserve  officer  who  had  been  a teacher  in  civOian  life.  This  wise  gentleman  discarded 
all  reference  to  educational  theory  and  made  practice  taaching  the  heart  of  his  course. 

He  r^uired  each  new  instructor  to  plan  and  present  one  short  lesson  and  one  long  lesson, 
of  the  kind  they  would  actually  be  presenting  on  the  job.  The  new  program  ran  four 
hours  a day  for  two  weeks  and,  when  the  training  phase  ended,  the  program  included 
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irregularly-scheduled  monitoring  of  the  new  instructors  in  their  classrooms.  Other  Army 
schools  followed  this  example  with  such  success  that  the  Armored  School  became,  and 
remains  to  this  day,  the  proponent  Army  agency  for  “methods  of  instruction”  matters. 

v'ollowing  the  War,  the  American  Council  on  Education  established  a Commission 
on  Implications  of  Armed  Services  Educational  Programs.  Some  258  educators  who  had 
served  in  the  Armed  Forces  were  asked  the  most  important  features  of  armed  services 
training.  While  there  was  considerable  agreement  on  the  ten  top  features,  there  was  almost 
unanimous  apeement  (92%)  that  the  single  most  out'  tending  feature  was  use  of  visual 
aids  in  military  training.  (6)  In  fact,  it  was  estimate  hat  the  quantity  of  audio-visual 
aids  created  and  used  by  the  Army  and  Navy  betwee  1940  and  1945  was  six  times  the 
quantity  of  similar  material  created  for  use  in  all  civilian  education  in  all  earlier  years.  (7) 

Training  Research 

For  the  element  of  training  research,  we  must  turn  our  attention  to  the  Army  Air 
Forces  and,  .specifically,  to  the  Army  Air  Forces  Aviation  Psychology  Propam.  Even  an 
amateur  historian  knows  better  than  to  assert  a precise  date  for  any  long-time,  continuing 
trend,  but  one  possible  date  for  the  beginning  of  scientific  training  research  in  the  Army 
would  be  July  15,  1941.  On  that  date.  Dr.  John  C.  Flanagan,  Associate  Director  of  the 
Cooperative  Test  Service,  was  commissioned  a major  in  the  U.S.  Army  and  called  to  active 
duty  to  establish  a psychological  agency  in  the  U.S.  Army  Air  Corps  (later  to  become  the 
U.S.  Army  Air  Forces).  He  reported  for  duty  in  Washington,  D.C.,  the  following  day  to 
set  up  plans  for  developing  and  validating  a battery  of  printed  and  apparatus  tests  for  use 
in  selecting  pilots.  (8) 

Major  Flanagan  moved  promptly.  Within  a month  he  had  sketched  the  outlines  of 
a propam  and  had  obtained  commitments  of  cooperation  from  the  Air  Corps  Training 
Division.  He  established  a Psychological  Research  Unit  at  Maxwell  Field,  Alabama,  to 
develop  tests  of  personality  and  temperament.  This  unit  was  headed  by  Laurance  F. 

Shaffer.  He  established  a second  research  unit  at  Kelly  Field,  Texas,  under  Robert  T.  P.ock, 
to  develop  psychomotor  tests.  He  also  established  a third  research  unit  at  Santa  Ana, 
California,  under  J.P.  Guilford,  to  develop  printed  tests  of  intellectual  functions. 

Over  the  next  4-1/2  years,  the  Air  Forces  Aviation  Psychology  Program  utilized  the 
ser/ices  of  some  1G5  officers,  1,255  enlisted  men,  and  25  enlisted  women  (WaCs).  It  was. 
at  the  time,  the  lai^est  assemblage  of  psychologists  ever  to  work  on  a common  problem- 
improving  the  personnel  system  of  the  Army  Air  Forces.  It  is  an  obvious  injustice  to  the 
^ names  of  propam  participants.  I do  so  here  only  to  give  readers 
a “feci”  for  the  cailber  of  personnel  involved  in  the  propam.  Among  them  were:  Judson 
Brown;  Urie  Bronfenbrenner;  Launor  F.  Carter;  Meredith  P.  Crawford;  John  T.  Dailey; 

Philip  H.  DuBois;  Stanford  C.  Ericksen;  I.E.  Farber;  Glen  Finch;  N.L.  Gage;  Robert  M. 
Gagne;  Robert  Glaser;  Frank  A.  Geldard;  Edwin  E.  Ghiselli;  Walter  F.  Grether;  John  K. 
•HemphiU;  Roger  W.  Heyns;  Nicholas  R.  Hobbs;  Paul  Horst;  Lloyd  G.  Humphries;  Edward  H. 
Kemp;  Joseph  T.  Klapper;  William  A.  McClelland;  Arthur  W.  Melton;  Neal  E.  Miller; 

Gabriel  D.  Ofeish;  Henry  W.  Riecken,  Jr.;  Roger  W.  Russell;  Benjamin  Shimberg; 

Lawrence  M.  Stolurow;  Robert  L.  Thorndike;  J.E.  Uhlaner;  T.R.  Vallance;  and  S.  Raines 
Wallace. 

As  Dr.  Robert  L.  Thorndike  pointed  out  after  the  War,  the  Aviation  Psychology 
Program  actually  went  through  tw'o  phases:  first,  the  development  of  selection  and 
classification  tests  and  procedures  and,  later,  attention  to  other  psychological  aspects  of 
militoy  effectiveness.  (9)  One  major  activity  in  this  second  phase  was  research  on 
training-primarily  training  for  pilots,  navigators,  and  bombardiers,  but  also  training  for 
other  aircrew  members.  Space  does  not  permit  any  discussion  of  the  substance  of  this 
research  but,  thanks  to  the  wisdom  and  persistence  of  Dr.  Flanagan,  the  written  record 
survives. 


During  the  war,  most  of  the  work  of  this  extensive  program  was  known  only  within 
the  military.  “National  security”  kept  program  participants  from  traditional  publication 
in  scientific  journals,  and  the  limited  copies  of  their  research  reports  began  disappearing 
as  the  war  drew  to  a close.  With  the  cooperation  of  a number  of  his  wartime  colleagues. 
Dr.  Flanagan  conceived  and  shepherded  through  the  U.S.  Government  Printing  Office  a 
19-volume  series  of  Army  Air  Forces  Aviation  Psychology  Program  Research  Reports, 
published  in  1947.  When  I stumbled  into  the  military  training  research  field  in  1963, 
this  series  of  reports  was  already  long  out  of  print.  A few  years  later,  in  my  own  major 
contribution  to  the  field,  I was  able  to  persuade  the  Defense  Documentation  Center 
(which  does  not  normally  accept  materials  published  by  the  USGPO)  to  make  an  exception 
in  the  case  of  these  research  reports.  The  complete  19-volume  set  is  now  permanently 
available  in  hard-copy  and/or  microfiche  from  both  DDC  and  the  National  Technical 
Information  Service  (NTIS).  The  record  is  preserved! 

The  Post-War  Period 

When  World  War  II  ended  most  of  the  psychologists  in  the  Aviation  Psychology 
Program  traded  in  their  uniforms  for  mufti  and  returned  to  their  university  classrooms 
and  laboratories.  Those  who  wanted  to  continue  in  aviation  psychology  were  able  to  do 
so  in  the  United  States  Air  Force,  which  came  into  its  own  in  1947  as  a Service  separate 
and  apart  from  the  Army.  The  leaders  of  this  new  Service  quickly  demonstrated  their 
conviction  that  psychology  had  much  to  offer  the  Air  Force  by  creating,  in  rapid  suc- 
cession, the  Human  Resources  Research  Center  within  the  Air  Training  Command;  the 
Human  Resources  Research  Laboratories  (later  called  the  Human  Factors  Operations 
Research  Laboratories)  within  Headquarters  Command,  USAF;  and  the  Human  Resources 
Research  Institute  in  the  Air  University  Command.  In  1954,  these  laboratories  were  all 
transferred  to  the  Air  Research  and  Development  Command,  which  merged  them  into  its 
own  Air  Force  Personnel  & Training  Research  Center  (AFPTRC).  But  that  is  a separate 
story  and,  for  us  following  the  path  of  Army  training  research,  it  is  the  Nvrong  fork  in 
the  road— worthy  of  following  in  its  own  right,  but  unable  to  take  us  where  we  want  to 
go  today. 

The  Personnel  Research  Section  in  the  Army  Adjutant  General’s  Office  had  contin- 
ued working  tiuoughout  the  World  War  II  demobilization  period,  primarily  in  its  tradi- 
tional areas  of  selection,  classification,  and  assignment.  In  1947,  some  members  of  the 
Section  turned  their  attention  to  problems  of  military  training— specifically,  to  problems 
of  measuring  the  results  of  basic  training  and  of  career  schooling.  However,  this  work 
represented  only  a small  proportion  of  the  Section’s  efforts,  which  continued  to  focus 
on  psychological  testing. 

As  1950  rolled  around,  the  Army  vras  still  not  actively  engaged  in,  or  supporting 
much  research  on  training.  The  situation  was  described  by  an  unidentified  author  in 
an  Army  staff  study,  “Human  Resources  Research  Program  within  the  Department  of 
the  Army,”  dated  June  22,  1950: 

Applicable  regulations  are  at  variance  in  placement  of  responsibility 
for  supervision  of  research  and  development  activities  within  the  Department 
of  the  Army.  . . . The  responsibility  for  implementation  of  research  and 
development  in  the  field  of  human  resources  is  widely  scattered  among  (a)  the 
various  Technical  Services;  (b)  the  Personnel  Research  and  Procedures  Branch, 

Office  of  the  Adjutant  General;  (c)  Office,  Assistant  Chief  of  Staff,  G-2; 

(d)  Research  and  Development  Division,  Office,  Assistant  Chief  of  Staff,  G-4; 
and  (e)  other  agencies  which  may  be  assigned  the  responsibility.  . . . The 
Department  of  the  Army  has  not  at  any  time  provided  personnel  in  sufficient 
quantity  and  quality  to  give  General  Staff  supervision  to  research  in  human 
resources  in  any  degree  comparable  to  that  provided  in  either  the  Navy  or  the 
Air  Force.  . . . The  annual  budget  allocations  for  research  in  .human  resources 
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in  the  Department  of  the  Army  has  been  1%  or  less  of  the  total  research  and 
development  budget  for  the  entire  Department  of  the  Army.  . . . there  has 
been  at  no  time  sufficient  staff  to  give  attention  to  the  needs  of  the  Depart- 
ment or  to  do  the  necessary  overall  and  long-range  planning  prerequisite  to 
the  development  of  an  adequate  program.  . . . Human  resources  research  is 
much  more  likely  to  be  dealing  with  intangibles,  and  the  end  products  are 
more  difficult  to  appraise.  For  these  reasons,  it  is  more  subject  to  curtail- 
ment in  times  of  budgetary  or  appropriation  shortages. 

Later  that  year,  another  staff  study,  “An  Army  Program  in  Human  Resources 
Research”  (November  28,  1950)  concluded  that  “Human  resources  research  can  provide 
valuable  assistance  to  the  Army  by  producing  and  evaluating  practical  methods  and  tech- 
niques which  will  increase  the  effectiveness  of  human  beings  in  military  operations.” 

The  study  recommended  establishment  of  a Training  Research  Unit  at  some  Army  post 
where  there  was  both  a division  and  other  tactical  units  being  trained,  and  if  this  unit 
proved  successful,  others  would  be  established  elsewhere  in  the  Army. 

On  January  12,  1951,  another  staff  study,  entitled  “Responsibility  for  General  Staff 
Supervision  of  Human  Resources  Research,”  recommended  that  conflicting  Army  Regula- 
tions be  Avritten  to  place  this  responsibility  solely  with  the  Research  and  Development 
Division,  Office  of  Assistant  Chief  of  Staff,  G-4.  (The  responsibility  actually  was  vested 
later  that  year  with  the  Office  of  Assistant  Chief  of  Staff,  G-1.) 

How  far  “up  the  channel”  any  of  these  studies  actually  floated  is  unknown  to  me, 
and  is  important,  perhaps,  only  insofar  as  they  were  precursors  to  a staff  study  that  did 
get  action,  and  that  did  put  the  Army  solidly  into  training  research.  That  crucial  study, 

“An  Integrated  Program  in  Human  Resources  Research,”  was  written  primarily  by  Dr,  Harry 
Harlow,  who  then  worked  in  the  G-4  Research  and  Development  Division  as  the  U.S.  Army 
Chief  Psychologist.  It  was  submitted  to  Army  Chief  of  Staff  J.  Lawton  Collins  on 
June  7,  1951.  Among  its  recommendations  were  the  following: 

9.  That  a major  contract  be  awarded  to  a recognized  educational 
institution  to  provide  for  the  formation  of  a Human  Resources  Research 
Office,  which  would  have  primary  responsibility  for  conducting  research 
in  the  areas  of  training  methods,  motivation  and  morale,  and  psychological 
warfaie  techniques. 

10.  That  the  Human  Resources  Research  Office  carry  out  its  responsi- 
bility by; 

a.  Conducting  researches  at  an  established  central  office. 

b.  Granting  and  monitoring  contracts  to  appropriate  educational, 
business,  ard  industrial  organizations. 

c.  Providing  the  civilian  staff  for  in-service  research  units  and 
furnishing  technical  supervision  of  research  conducted  at 
military  installations. 

11,  That  appropriate  research  units  be  established  at  selected  military 

installatio.iE  to  give  primary  research  emphasis  to  the  following 

areas: 

a.  Training  methods, 

b.  Motivation  and  morale. 

c.  Psychological  warfare. 

Two  weeks  later.  General  Collins  called  Dr.  Harlow  into  his  office  for  a briefing  on 
the  proposal  and  for  a question-and-answer  session.  When  he  had  satisfied  himself  on 
the  soundness  of  the  plan.  General  Collins  transmitted  the  study  with  its  proposals  and 
his  endorsement  to  the  Secretary  of  the  Army. 

On  July  27,  1951,  the  Army  entered  into  a contract  with  The  George  Washington 
University  for  the  establishment  of  a Human  Resources  Research  Office  (HumRRO)  to 


carry  this  concept  into  action.  Dr.  Meredith  P.  Crawford,  who  had  headed  research 
units  in  the  Army  Air  Forces  Aviation  Psychology  Program  and  who  was  then  Dean  of 
the  College  of  Arts  and  Sciences  at  Vanderbilt  University  was  invited  to  come  to 
Washington  to  head  this  new  Office  and  program. 

From  this  beginning,  Dr.  Crawford  and  his  colleagues  in  HumRRO  have  provided  the 
Army— over  the  past  quarter-century— with  a training  research  program  that  has  brought 
credit  to  the  Army  and  to  themselves.  HumRRO  scientists,  primarily  psychologists, 
have  been  effective  in  helping  to  move  the  Army  to  its  present  stage  of  extremely 
effective,  and  still  improving,  training  and  education  programs. 

The  work  that  begain  in  the  Aviation  Psychology  Program  in  World  War  II  and  con- 
tinued in  the  Army  under  HumRRO  has  demonstrated  that  consistent  improvement 
in  educational  methods  and  practice  can  be  based  on  a systematic  application  of  the 
methods  of  psychological  science.  By  1951,  with  the  establishment  of  the  Human  Resources 
Research  Office,  the  Army  was  on  its  way  toward  the  development  of  a practical  tech- 
nology of  instruction. 
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ATC-PERFORM: 


A CASE  STUDY  IN 

INDIVIDUAL  MILITARY  TRAINING  RESEARCH  AND  DEVELOPMENT 

H.H,  McFann  and  J.E.  Taylor 


Background  and  History 

This  paper  describes  the  impact  on  the  operational  Army  of  an  applied  research  and 
development  effort  undertaken  by  HumRRO  under  the  code  name  ATC-PERFORM.  A 
bit  of  history  is  required  to  place  this  activity  in  perspective. 

Since  1952,  HumRRO  has  been  involved  in  working  with  the  Army  on  projects  related 
to  training  center  operations.  In  the  early  T960s,  the  Army  and  HumRRO  mutually 
agreed  that  one  of  the  HumRRO  Divisions  should  be  involved  exclusively  in  research  and 
development  with  Army  Training  Centers.  Prior  to  this,  research  and  development  efforts 
involving  Army  Training  Center  activities  had  consisted  of  discrete  activities,  not  neces- 
sanly  related,  which  focused  on  specific  aspects  of  Training  Center  operations.  They 
resulted  in  package  programs  that  could  be  inserted  into  the  existing  operational  framework. 
Although  these  projects  had  resulted  in  meaningful  change,  their  potential  impact  was 
somewhat  limited.  It  was  felt  that  devoting  a large  effort,  over  an  extended  time  period, 
to  the  total  training  center  system  would  magnify  the  potential  impact  and  contribution 
of  training  research  and  development. 

From  the  start,  the  Army  and  HumRRO  were  in  agreement  on  both  the  area  of  con- 
centration and  the  cooperative  nature  of  the  venture.  The  Army  would  support  the 
research,  providing  ready  access  to  the  personnel  and  activities  of  a Training  Center. 
HumRRO  would  devote  an  agreed-upon  level  of  effort  to  this  joint  venture,  providing 
technical  advisory  service  to  the  Army  as  well  as  mmntaining  an  integrated  research  and 
development  program  focused  on  Army  Training  Center  problems.  Thus,  the  intent  from 
the  start  \va.s  to  move  through  iterations  of  research,  development,  testing,  evaluation  and 
utilization  cycles  in  order  to  make  change  in  Army  training  operations  from  a sound 
empirical  data  base. 

From  1961  to  1966,  HumRRO  undertook  an  integrated  program  of  research  and 
development  aimed  at  studying  the  various  components  of  overall  Army  Training  Center 
operations.  Topics  included  activity  analysis,  management  and  leadership  practices,  instruc- 
tional content  and  methods  employed,  evaluation  procedures,  and  characteristics  of  trainee 
input  and  cadre.  Numerous  products  evolved  which  guided  future  research  and  development 
efforts  and  resulted  in  changes  in  existing  practices;  e.g.,  the  Drill  Sergeant  Program,  the 
Leader  Preparation  Program,  moving  from  a single  evaluative  instrument  for  assessing  BCT 
proficiency  to  multiple  alternate  forms,  and  individual  course  revisions. 

In  about  mid-1966,  a major  policy  decision  was  made  which  affected  the  direction 
of  the  research  and  development  activities  as  well  as  the  operation  of  the  Training  Centers. 
Project  “One  Hundred  Thousand”  was  initiated  which  dramatically  increased  the  input 
of  lower  mental  aptitude  (/.'  yr  Category  IV)  recruits  into  the  Army's  training  population. 
HumRRO  undertook  a series  of  studies  to  determine  what  effect  this  policy  change  would 
have  on  training  and  utilization  of  perso.mel.  Examination  of  the  then  current  instruc- 
tional system  for  Basic  Combat  Training  and  Advanced  Individual  Training  as  conducted  in 
Training  Centers  indicated  that  the  system  was  hard  pressed  to  carry  this  new  input. 
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Generally  speaking,  the  system  at  that  time  could  be  characterized  as  a fixed-paced  group 
system,  relying  primarily  on  a passive  classroom-lecture  instructional  paradigm  which 
utilized  normative  evaluation  procedures  with  a heavy  reliance  on  paper-and-pencil  testing. 
This  system  had  worked  reasonably  well  for  higher  aptitude  groups,  and  it  could  cope 
with  the  small  number  of  lower  mental  ability  personnel  who  had  historically  been  in  the 
system.  But  the  magnitude  of  this  new  trainee  input  over-extended  the  system. 

A further  training  problem  occurred  when  the  policy  decision  was  made  to  do  away 
with  college  deferments.  The  result  was  a training  load  characterized  by  a very  wide 
spread  of  individual  ability,  ranging  from  functional  illiterates  to  college  graduates.  The 
fixed-paced,  group-lecture  mode  employed  in  the  Army  Training  Centers  could  not  handle 
this  diversity  of  personnel.  The  more  able  people  were  lost  to  boredom  and  the  less  able 
were  often  lost  to  attrition. 

From  about  1968  to  1972,  this  HumRRO  research  and  development  effort  concen- 
trated on  conceptualizing,  developing,  and  testing  instructional  strategies  for  training  per- 
sonnel across  this  wide  ability  spectrum.  Individual  differences,  as  related  to  training  and 
performance,  became  paramount  areas  of  interest.  Research  and  development  activities 
included  laboratory  studies  concerned  with  content,  instructional  approach,  and  aptitude 
level;  development  and  test  (in  an  operation  context)  of  instructional  systems  geared  to 
handle  the  heterogeneous  training  population;  provision  of  consulting  services  to  Army 
personnel  in  their  revision  of  the  heavy  density  AIT  combat  and  combat  support  courses; 
and  determination  of  how  various  aptitude  personnel  performed  ou  the  job. 

In  the  spring  of  1970,  the  Department  of  the  Army  undertook  a broad  and  long- 
range  plan  for  conversion  to  the  All-Volunteer  Army.  HumRRO  was  requested  to  help 
formulate  the  master  plan  for  guiding  training  innovations,  and  in  conjunction  with  Fort 
Otd,  to  develop  and  test  the  Experimental  Volunteer  Army  Training  Program  (EVATP) 
which  was  to  serve  as  a model  for  all  Army  Training  Centers.  This  massive  developmental 
undertaking  involved  all  aspects  of  the  Training  Center:  organization,  management,  instruc- 
tional systems,  instructor  training,  and  evaluation.  All  of  the  previous  research  and  dfevel- 
opment  activities  served  as  a springboard  for  this  “cold  water”  bath  of  reality  testing. 

It  was  here  that  the  previously  researched  concepts  of  performance-oriented  instruction  • 
and  criterion-referenced  testing  were  subjected  to  large-scale  experimental  field  evaluation. 
Of  equal  importance,  it  was  during  this  implementation  phase  that  the  problems  of  resist- 
ance to  institutional  change  became  crystal  clear.  However,  the  solutions  to  such  problems 
remained  somewhat  murky. 

In  April  1971,  the  U.S.  Continental  Army  Command  (now  the  U.S.  Army  Training 
and  Doctrine  Command-TRADOC)  directed  that  all  training  programs  conducted  in  Army 
Training  Centers  be  reviewed  and  revised  in  accordance  with  systems  engineering  of  train- 
ing and  the  principles  of  the  EVATP,  performance-oriented  instruction  and  testing. 
HumRRO  technical  assistance  was  to  be  used  during  revision  or  redesign  of  instruction. 
From  1971  through  June  1972,  HumRRO  personnel,  in  addition  to  working  with  Fort  Ord 
personnel  on  further  refinement  of  the  performance-oriented  testing  and  training  program, 
became  actively  involved  in  the  accomplishment  of  such  system  change.  Activities  involv^ 
consultation  in  the  development  of  implementation  guidance  documents  and  a large  num- 
ber of  orientation  visits  to  training  centers  and  Army  schools  to  promulgate  the  concepts 
and  techniques  of  performance-oriented  training  and  testing.  Direct  assistance  was  pro- 
vided to  the  training  staffs  of  AIT  combat  support  courses  (e.g..  Cook,  Wheel  Vehicle 
Mechanic,  and  Radio  Operator)  in  preparing  performance  tests  and  modifsring  training 
techniques  for  these  courses.  Guidance  was  provided  in  modifying  NCO  Academy  and 
Drill  Sei^eant  programs  to  insure  not  only  that  these  principles  were  incorporated  into 
the  content  of  their  training  programs  but  also  that  their  course  of  instruction  was  based 
upon  them. 
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By  spring  1972,  the  training  concepts  had  been  implemented  in  all  CONUS  Army 
Training  Centers  where  Basic  Combat  Training  and  Advanced  Individual  Training  in  the 
combat  military  occupational  sjsecialities  were  being  conducted. 


ATC-PERFORM 

Such  is  the  history  and  background  which  spaw'ned  ATC-PERFORM,  an  effort  that 
was  to  extend  over  the  years  1972-1975.  Its  specific  objective  was  to  provide  technical 
research  and  development  assistance  to  ^Vrmy  agencies  involved  in  the  review,  evaluation, 
and  refinement  of  performance-based  training  techniques  in  Army  Training  Centers.  It 
extended  the  Army’s  effort  to  accomplish  major  training  innovations  that  had  been 
initiated  with  the  EVATP  in  1971-72  during  conversion  to  an  all-volunteer  status. 

Research  and  development  assistance  was  provided  in  such  diverse  activities  as  task  anal- 
ysis, performance  test  development,  instructional  systems  development,  conduct  of  evalua- 
tions, design  of  experiments  on  the  effects  of  instructional  innovations,  collection  and 
analysis  of  questionnaires  and  inter\new  data,  generating  or  revision  of  training  literature, 
and  orientation  of  training  managers  and  instructors. 

Examination  of  these  myriad  activities  gives  some  idea  as  to  the  scope  and  nature  of 
this  effort.  From  an  instructional  technology  perspective,  emphasis  was  placed  on  further 
codification  of  the  principles  developed  and  tested  under  the  rubric  of  performance-based 
training  and  testing,  and  on  extending  this  instructional  technology.  Equally  important 
was  the  accomplishment  of  institutional  change.  Actions  were  taken  to  involve  Army  per- 
.sonnel  in  the  conversion  of  specific  courses  and  programs  of  instruction,  and  to  make 
certain  that  policies  and  procedures  were  established  insuring  that  the  gains  achieved  in 
instructional  effectiveness  and  efficiency  through  application  of  the  principles  would  become 
institutionalized. 

The  principles  have  been  implemented  in  various  configurations,  in  the  following  Army 
Training  Center  Programs;  (a)  Basic  Training  (Basic  Combat  Training  for  Men,  Basic  Train- 
ing for  Women);  (b)  Advanced  Individual  Combat  Training  (Infantry,  Armor,  Field  Artil- 
lery, and  Air  Defense  Artillery);  (c)  Advanced  Individual  Combat  Support  Training  (Signal, 
Tr.il'.sportation,  Ordnar.<c,  Clerical,  Quartenna.=ter,  Military  Police,  and  Engineai),  (d>  NCO 
^eadcrsiiip  and  in.struclor  training  courses;  and  (e)  training  for  Reserve  Components.  The 
institutionalizing  of  the.se  concepts  and  techniques  is  reflected  in  a variety  of  Army  train- 
ing documents  including  Army  Subject  Schedules,  Programs  of  Instruction,  and  Army 
Training  Programs;  in  a number  of  pamphlets  in  the  TRADOC  600  Serie.s;  in  instruction 
and  asses-sment  materials  for  a wide  variety  of  specific  courses  of  instruction,  and  in  Army 
staff  policy  decisions. 

The  concepts  of  One  Station  Unit  Training  (OSUT)  and  One  Station  Training  (OST), 
which  involve  the  combining  of  Basic  and  Advanced  Individual  Training,  grew  out  of  the 
EV.ATP  and  this  program  of  research.  Recent  legislation  based  partly  on  ATC-PERFORM 
data,  permits  training  time  to  be  reduced  to  12  weeks.  This  modifies  a Public  Law  that 
had  been  enacted  during  the  Korean  War  specifying  that  training  be  at  least  16  weeks. 

A number  of  courses  previously  mandated  to  be  16  weeks  m length  are  being  selectively 
reduced  by  as  much  as  four  weeks  (e.g.,  armor  crewman,  light  weapons,  infantrymen, 

.Military  Police,  telephone  lineman). 

As  part  of  ATC-PERFORM  activities,  experimental  systems  of  self-paced  instruction 
were  tested  out  in  supply  (cognitive)  and  heavy  equipment  (motor  skill)  courses  which 
had  lieen  previously  converted  to  performance-based  training  and  testing.  The  results 
iiidicafed  that  self- pacing  in  combination  with  performance  training  is  feasible,  practical, 
highly  effective,  and  provides  the  potential  for  accelerating  individuals  through  the  train- 
ing Irase.  (Savings  of  some  25%  in  training  time  were  achieved  with  no  decline  in  skill 
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proficiency.)  Drawing  upon  this  research,  TRADOC  is  directing  that  Army  MOS- 
producing  courses  be  based  upon  performance  training  principles  and  that  they  include 
self-pacing.  Thus,  the  operational  impact  of  this  research  will  likely  continue  for  at  least 
the  next  two  or  three  years. 

Research  and  development  has,  throughout,  attempted  to  remain  responsive  to 
major  policy  decisions  such  as  Project  One  Hundred  Thousand,  the  All-Volunteer  Army, 
and  the  Army’s  draw-down  in  funds  and  resources.  The  goal  has  been  to  find  ways  to 
insure  that  Army  training  prepared  the  spectrum  of  trainees  for  their  Army  assignments 
in  the  most  efficient  manner  possible,  efficiency  being  defined  as  the  ratio  of  skill  profi- 
ciency to  training  costs.  Accomplishment  of  this  goal  has  requried  continuous  study  of 
the  complete  job  training  system  (course  development,  course  structure,  course  manage- 
ment, and  training  methods). 

In  summsoy,  ATC-PERFORM  has  been  a major  catalyst  for  accomplishing  institutional 
change  in  the  instructional  system  used  in  Army  Training  Centers.  The  change  has  affected 
training  managers,  instructors,  and  trainees.  Change  in  instructional  methods  has  been 
directed  away  from  the  platform,  and  subject-oriented  systems  to  a performance-oriented 
system.  Training  objectives  that  facilitate  an  individual’s  learning  entry  level  job  skills 
are  emphasized.  Training  is  focused  on  the  individual  rather  than  on  the  group.  The 
instructor  has  become  a manager  and  organizer  of  skill  practice  rather  than  a presenter 
of  verbal  instruction. 

Ample  evidence  was  obtained  to  confirm  that  institutional  change  in  large  organiza- 
tions does  not  take  place  easily  or  quickly.  While  trainees  adapt  readily  to  this  new  system 
and  find  performance-oriented  training  both  meaningful  and  motivating,  the  process  of 
introducing  change  through  training  managers  and  instructors  who  are  products  of  the 
“conventional”  system  takes  time  and  effort.  It  is  apparent  that  institutional  change,  if 
it  is  to  occur  and  be  maintained,  requires  supervision  from  training  managers  and  com- 
manders; hence,  their  behavior  must  change  also. 

It  should  also  be  apparent  by  now,  that  an  undertaking  such  as  ATC-PERFORM 
requires  mutual  understanding,  cooperation,  patience,  and  considerable  tolerance  for 
frustration  among  all  participants— researchers,  sponsors,  instructional  personnel,  and 
trainees. 
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THE  ROLE  OF  SIMULATORS  IN  PILOT  TRAINING: 
SOME  OUESTIONS  REGARDING  EFFECTIVENESS 

Paul  W.  Caro 


Introduction 

The  use  of  simulators  and  other  training  devices*  by  the  military  has  grown  rapidly 
in  recent  years.  Whether  this  phenomenon  is  attributable  to  the  desire  to  improve  train- 
ing efficiency  and  effectiveness  or  to  other  causes  is  an  open  question.  Following  a study 
of  the  use  of  aircraft  simulators  in  selected  U.S.  military  and  civilian  pilot  training  pro- 
grams, the  Comptroller  General  of  the  United  States  recently  issued  a report  to  the  Con- 
gress which  was  critical  of  the  extent  to  which  simulators  are  being  used  in  the  military 
training  programs  studied.  (1)  The  report  suggested  that  present  knowledge  of  simulator 
design  and  employment  is  sufficient  to  support  much  more  extensive  use  of  simulators 
than  was  typically  found  to  be  the  case.  The  report  cites  certain  pilot  training  agencies 
which  seem  to  employ  that  knowledge  more  effectively  than  do  others,  even  though  the 
knowledge  is  available  to  all. 

The  Comptroller  General’s  report  was  not  intended  as  a scientific  study  of  psycho- 
logical factors  and  their  influences  on  simulator  training  effectiveness.  Instead,  it  identified 
factors  related  to  the  management  of  simulator  training  and  attitudes  toward  such  training 
which  tend  to  impede  more  extensive  use  of  simulators.  Factors  identified  include  regula- 
tions emphasizing  aircraft  rather  than  simulator  training,  inadequate  instructor  training, 
failure  to  use  simulator  capabilities  fully,  and  poor  simulator  maintenance.  The  influence 
of  such  management  factors  upon  trainer  use  has  been  documented  in  the  research  litera- 
ture. (2,  3)  Other  factors  which  have  been  identified  as  impediments  to  effective  simulator 
use  include  the  design  of  the  simulators  themselves  and  the  design  of  the  training  programs 
in  which  the  simulators  are  employed.  (4,  5)  Still  other  factors  have  been  reported  that 
Lnfiuence  the  acceptance  of  simulators  by  those  who  must  use  them  and  thereby  ifidirectly 
impede  their  more  extensive  use.  (6) 

Extensive  use  of  simulators  is  not  necessarily  synonymous  with  effective  training, 
however.  In  one  study  (7),  it  was  found  that  the  use  of  a particular  device  added  cost, 
but  no  training  value,  to  an  already  expensive  pilot  training  program.  Another  study 
demonstrated  that  the  training  value  of  a device  could  be  increased  substantially  without 
increasing  the  amount  of  device  training  time  when  the  manner  in  which  the  device  was 
u-sed  was  changed.  (8) 

While  there  is  a great  deal  of  scientific  and  training  literature  in  existence  dealing  with 
simulator  training,  some  of  the  more  significant  factors  influencing  transfer  of  simulator 
training  have  not  received  the  attention  they  warrant.  The  purpose  of  this  paper  is  to  call 


•Many  writers  distinguish  between  simulators  (e.g.,  “ . . . a high  degree  of  relevance  to  operational 
equipment . . .”)  and  training  devices  (e.g.,  “ . . . any  piece  of  apparatus  which  is  used  for  training  . . .”). 
Tlie  present  writer  will  treat  these  examples  of  training  equipment  as  members  of  a single  class.  To 
quote  Gagne  (9,  p.  96):  “What  distinguishes  a training  device  (from  other  training  equipment)  is  not 
it?  appearance  or  construction,  but  rather  how  and  for  what  purposes  it  is  used.”  The  term  simulator  is 
used  here  to  identify  ground-based  training  equipment  used  for  the  purpose  of  training  pilots  to  fly  air- 
craft. The  question  of  how  it  is  used  will  be  addressed  in  the  paper. 
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attention  to  some  of  these  factors  and  to  note  their  impact  on  transfer  effectiveness.  The 
emphasis  will  be  upon  increasing  the  effectiveness  of  simulator  training  as  opposed  merely 
to  increasing  simulator  use.  While  simulator  effectiveness  and  use  obviously  are  not  inde- 
pendent (a  simulator  cannot  be  used  effectively  when  it  is  used  little  or  not  at  all),  they 
are  not  always  coincident.  Some  simulator  designers  and  users  appear  unaware  of  factors 
which,  if  properly  treated  or  mariaged,  would  markedly  enhance  the  value  and  efficiency 
of  simulator  training  programs.  The  unfortunate  consequence  of  their  lack  of  awareness 
that  simulator  training  effectiveness  suffers:  skills  which  may  be  critical  to  safe  opera- 
tion of  an  aircraft  may  not  be  developed,  and  aircraft  scheduled  for  training  purposes  may 
be  needed  elsewhere. 

Factors  Influencing  Simulator  Training  Effectiveness 

A recent  Summary  of  simulator  training  studies  (3)  indicated  that  simulator  training 
effectiveness  has  increased  markedly  since  World  War  II.  Some  of  the  increase  can  be 
attributed  to  advances  in  engineering  and  instructional  design  technologies.  Simulation 
engineers  now  have  the  technology  available  to  build  simulators  which  more  nearly  satisfy 
Thorndike’s  common  elements  design  hypothesis,  and  instructional  system  designers 
have  learned  how  to  zero  in  on  tasks  to  be  trained.  But  much  of  the  increase  has  come 
about  as  a result  of  research  and  experience  with  simulators  in  operational  training  settings. 

In  spite  of  the  large  number  of  simulator  effectiveness  studies  that  have  been  com- 
ideted,  there  remains  much  to  be  learned  about  training  \vith  these  devices.  A number  of 
factors  have  been  nominated  as  influences  upon  simulator  training  effectiveness,  but  many 
of  those  nominations  have  been  based  on  inference  rather  than  experimental  evidence. 

In  their  review  of  simulator  research.  Muckier,  Nygaard,  O’Kelley,  and  Williams  (14)  noted 
that  many  studies  compound  the  influence  of  several  potential  inftuences  (such  as  train- 
ing program  content,  instructional  technique,  and  instructor  qualification)  into  a single 
independent  variable  so  that  the  transfer  effects  can  be  attributed  only  to  the  unique  com- 
bination of  those  influences.  Even  in  the  few  experimental  investigations  which  isolate 
assumed  influences,  the  results  must  be  interpreted  cautiously  because  they  address  unique 
training  requirements  and  have  not  been  replicated. 

The  methodological  problems  involved  in  identifying  factors  which  influence  simulator 
training  effectiveness  cannot  easily  be  overcome.  Suspected  factors  can  seldom  be  exam- 
ined in  isolation.  It  is  difficult,  for  example,  to  determine  experimentally  the  relative 
value  of  a remote  instructor  station  vs.  an  on-board  or  in-the-cockpit  station  even  if  a 
suitably  designed  simulator  were  available  for  the  research,  because  to  use  each  station  to 
its  best  advantage  would  necessitate  having  two  methods  of  training:  one  optimized  for 
remoted  instruction;  the  other  optimized  for  on-board  instruction.  The  experiment  would 
thus  compare  instructor  station  and  training  program  combinations  rather  than  a simulator 
design  feature  in  isolation  from  other  factors.  The  training  program  factor  cannot  be 
held  constant.  It  would  be  inappropriate  to  compare  two  simulator  designs  using  a pro- 
gram optimized  for  only  one  or  for  neither. 

The  problem  of  generalizable  results  is  not  limited  to  studies  involving  operational 
simulators.  Even  using  equipment  designed  and  dedicated  to  research,  problems  arise. 

For  example,  to  pursue  the  illustration  of  instructor  .station  locarion  described  above, 
simulator  hardware  inflexibility  makes  it  difficult  to  conduct  the  necessary  research  lead- 
ing to  the  design  of  the  optimvim  remote  instructor  station  for  experimental  comparison 
with  the  optimally  designed  on-board  station.  Additionally,  the  on-board  station  design 
which  is  optimum  for  a single-seat,  high-performance  attack  aircraft  simulator  with  a 
visual  display  may  bear  little  resemblance  to  the  optimized  on-board  station  for  under- 
graduate instrument  training  in  a side-by-side  seating  helicopter  simulator. 

Definitive  data  do  not  exist  which  will  permit  the  quantification  of  the  influence 
of  all  factors  believed  to  bear  on  simulator  training  effectiveness.  In  fact,  the  mere 
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identification  of  most  such  factors  rests  upon  inference,  conjecture,  and  untested  hypoth- 
eses. The  absence  of  hard  data  obviously  cannot  justify  suspected  factors  being  ignored, 
however.  Where  inferences  can  be  made  and  supported  by  consensus,  factors  believed  to 
influence  simulator  training  must  be  taken  into  account  by  those  responsible  for  simulator 
design  and  use,  imless  evidence  can  be  assembled  to  refute  those  inferences.  Jt  is  the 
responsibility  of  the  research  commtmity  to  undertake  the  systematic  investigation  of 
such  factors. 

The  following  discussion  is  intended  to  call  attention  to  selected  factors  which 
influence  simulator  training  effectiveness.  It  would  not  be  fruitful  to  attempt  to  cite 
all  the  researchers  who  have  contributed  co  the  identification  of  such  factors.  Those  who 
have  contributed  to  the  literature  on  physical  vs.  psychological  variables  in  simulator  des^, 
for  example,  are  legion,  as  are  those  who  have  remarked  upon  the  importance  of  how  the 
simulator  is  used.  (Review  articles  touching  on  the  subject  include  references  2,  3,  14, 

15,  and  16.)  Except  where  reference  is  made  to  a particular  report,  the  author  acknowl- 
edges responsibility  for  the  inferences  set  forth  herein,  as  well  as  for  the  selection  of  fac- 
tors to  be  discussed. 


Simulator  Design 

There  are  two  areas  of  interest  with  respect  to  the  influences  of  simulator  design 
upon  transfer  of  training:  fidelity  of  simulation  and  design  for  training.  “Fidelity” 
refers  to  whether  features  of  the  aircraft  and  its  environment  are  included  in  the  simula- 
tor’s design,  and  the  extent  to  which  features  which  are  included  represent  or  duplicate 
their  real-world  counterparts.  “Design  for  training”  refers  to  the  inclusion  in  simulator 
design  of  featxues  or  configurations  which  facilitate  training  but  which  may  bear  no 
particular  resemblance  to  features  of  the  aircraft  and  environment  being  simulated. 

Fidelity  of  a simulation  is  often  equated  with  physical  correspondence  between  the 
device  and  its  real-world  counterpart.  In  their  discussion  of  simulator  design  considera- 
tions, however,  Smode  and  Hall  (17)  emphasize  instructional  strategies  and  capabilities 
and  suggest  that  fidelity  has  meaning  in  terms  of  the  process  and  the  realism  necessary 
to  promote  learning.  Design  characteristics,  they  assart,  should  be  defined  in  terms  of 
ass\iring  transfer  of  training.  In  other  words,  fidelity  of  simulation  is  a matter  of  the 
relevance  o^  the  simulation  to  the  training  objectives,  not  solely  a matter  of  physical 
corres]jondcnce.  This  concept  of  fidelity  accounts  for  the  effectiveness  of  so-called  low 
fidelity  devices  as  well  as  simulators  that  faithfully  reproduce  much  of  the  aircraft. 

Design  for  Training.  The  Smode  and  Hall  concept  of  fidelity  is  of  particular  interest 
with  respect  to  simulator  features  not  modeled  after  the  aircraft.  These  features,  which 
are  concerned  primarily  with  the  application  of  leamiii|  principles  to  the  training  process, 
include  freeze,  adaptive  training,  prompting  and  cueing,  performance  recording  and  play- 
back, performance  measurement,  and  various  instructor  station  displays  and  controls.  It 
is  generally  held  that  such  features  improve  the  conditions  under  which  learning  takes 
place  and  thereby  facilitate  the  attainment  of  training  objectives.  Therefore,  they  are 
factors  to  be  considered  in  judging  the  fidelity  of  a device  so  far  as  training  is  concerned. 

It  is  general  practice  to  adopt  innovative  simulator  design  features,  such  as  those 
mentioned  above,  on  the  basis  of  their  apparent  utility  without  subjecting  them  to  experi- 
mental scrutiny.  For  example,  the  widely  used  simulator  freeze  feature  was  implemented 
because  it  was  seen  as  an  aid  to  attaining  training  objectives  and  to  implementing  learning 
concepts  during  the  instructional  process.  Similarly,  other  design  decisions  are  made 
because  the  training  objectives  and  planned  concepts  of  simulator  employment  lead  to 
the  conclusion  that  a particular  design  is  appropriate  in  preference  to  others.  For  example, 
in  the  design  of  U.S.  Army  simulators  for  the  Vertol  CH-47  and  the  Bel!  AH-1  helicopters, 
the  instructor  stations  were  located  virtually  inside  the  cockpits  of  these  devices,  and 
certain  instructor  displays  were  positioned  so  that  they  could  be  viewed  by  both  the 
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instructor  and  the  trainees  in  order  to  facilitate  instructor-trainee  interactions  during  key 
training  activities.  The  training  effectiveness  of  these  features  probably  will  never  be 
determined  in  a transfer  experiment  for  reasons  advanced  elsewhere.  (See  Jeantheau  (10), 
Meister,  et  ai  (11),  and  Adams  and  McAbee  (12),  for  example.)  Analytically,  they  are 
believed  to  represent  effective  simulator  designs  with  respect  to  the  Army’s  training  pro- 
gram and  the  training  objectives  to  be  addressed. 

Visual  Fidelity.  Generally,  tasks  which  cannot  be  duplicated  or  even  approximated 
in  a device  cannot  be  learnt  there  for  subsequent  transfer  to  the  aircraft.  Tberefore, 
a simulator  in  which  more  tasks  characterizing  flying  can  be  performed  has  greater 
potential  training  effectiveness  than  one  in  which  fewer  such  tasks  can  be  performed. 

For  example,  a simulator  which  does  not  include  an  extra-cockpit  visual  display  would 
seem  to  have  less  effectiveness  potential  with  respect  to  training  tasks  requiring  visiud 
references  than  a simulator  with  such  a display. 

There  have  been  a number  of  studies  in  which  transfer  from  a simulator  with  a 
visual  display  has  been  demonstrated.  The  scenes  presented  by  some  of  these  displays 
are  much  simpler  than  scenes  viewed  from  an  airci^t.  For  example,  savings  in  aircraft 
time  required  to  perform  visual  reference  maneuvers  were  demonstrated  in  a study  by 
Flexman,  Matheny,  and  Brown  (18)  using  a simulator  with  a visual  display  consisting 
of  a line  drawing  on  a blackboard  placed  in  front  of  the  cockpit  and  tilted  by  an 
instructor  to  change  the  perspective  as  the  device  was  maneuvered  vrith  respect  to  sim- 
ulated ground  references.  The  effectiveness  of  other  simple  displays  consisting  of  styl- 
ized grids  and  lines  has  been  demonstrated  in  backward  transfer  situations  during  studies 
of  contact  analog  displays  developed  for  helicopters.  (19)  Displacement  of  scene  ele- 
ments consistirig  only  of  dots  and  lines  was  found  by  Thielges  and  Matheny  (20)  to 
provide  sufficient  information  for  the  performance  of  aircraft  control  tasks,  althou^ 
their  study  was  not  based  on  a transfer  model. 

These  studies  indicated  that  tasks  involving  aircraft  control  in  relation  to  extra-cockpit 
visual  information  can  be  practiced  effectively  in  simulators  vrith  very  simple  visual  scene 
displays.  The  displays  consisted  of  no  more  than  points,  lines  and  geometric  patterns 
arranged  in  accord  with  a set  of  mathematical  relationships  described  by  Gibson.  (21) 

Several  manufacturers  currently  are  taking  advantage  of  the  utility  of  these  simple  scene 
content  design  requirements  by  marketing  displays  which  represent  night  scenes  as  pat- 
terned points  of  light  on  a black  field,  and  their  displays  an*  being  used  with  apparent 
success  in  commercial  airlines’  simulator  training  programs. 

While  the  effectiveness  of  such  simple  visual  displays  has  been  demonstrated  to  the 
extent  described  above,  it  is  also  noted  that  simulators  without  a visual  display  can  be 
effective  in  the  training  of  visual  reference  flighi  tasks,  (n  a study  involving  a helicopter 
simulator  without  a display  or  any  other  representation  of  outside  visual  cues  except  the 
aircraft’s  navigation  and  attitude  instrument,  and  without  any  attention  during  simulator 
training  to  extra-cockpit  visual  cues  per  se,  stodents  trained  to  fly  instrum^t  fliglit  mis- 
sions in  the  device  qualified  in  the  aircraft  vmder  visual  conditions  more  rapidly  than  did 
students  not  receiving  the  prior  device  training.  (22)  In  a similar  study  using  a fixed  wing 
simulator  without  a visual  display,  a saving  in  visual  flight  time  required  to  complete  a 
transition  course  of  .approximately  50%  of  the  schedul^  course  length  was  obtained.  (8) 

An  unreported  study  by  the  U.S.  Air  Force  invohring  cognitive  training  in  a simulator 
with  no  visual  display  demonstrated  transfer  to  visual  flying  maneuvers  su.rh  as  traffic 
patterns.  (23) 

While  it  is  not  indicated  by  these  studies’  results  that  visual  displays  have  no  training 
value,  it  appears  that  many  behaviors  required  as  responses  to  extra-cockpit  visual  stimuli 
in  the  aircraft  can  be  practiced,  or  at  least  approximated,  in  response  to  stimuli  in  a sim- 
ulator without  an  outside  display.  Furthe.,  cockpit  instruments  provide  information  about, 
and  an  analog  display  of,  the  vi^al  world  outside  the  cockpit,  so  that  a pilot  flying 
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nstruments  is  responding  to  stimuli  analogous  to  those  available  to  the  pilot  flying  visually. 

At  least  some  of  the  simulator  effectiveness  attributed  to  the  simpler  visual  displays  pcoba- 
bly  would  occur  without  the  presence  of  such  a display  at  all. 

During  the  current  review,  no  studies  were  found  which  unequivocably  established  the 
effectiveness  of  any  extra-cockpit  visual  display.  While  transfer  studies  involving  visual  dis- 
plays were  found,  only  one,  an  exploratory  study  judged  inconclusive  by  its  authors  (24), 
included  a control  group  in  which  students  %vere  trained  in  the  simulator  without  using 
its  display.  Commercial  airlines  have  reduced  aircraft  training  time  following  the  addition 
of  a visual  display  to  an  existing  simulator,  but  some  if  not  all  of  the  reduction  resulted 
from  a priori  judgments  by  government  agencies  and  the  airlines  themselves  concerning 
increased  simulator  training  effectiveness.  In  no  cases  have  there  been  reports  of  efforts 
to  design  training  programs  that  would  seek  the  same  flight  training  savings  using  simulators 
wthout  visual  displays  that  presumably  have  been  achieved  using  simulators  with  such 
displays. 

The  lack  of  evidence  of  visual  display  training  effectiveness  cannot  be  taken  as  evidence 
of  their  lack  of  effectiveness.  There  is  a consensus  that  they  are  effective,  and  data  to 
contest  that  consensus  do  not  exist.  Logically,  it  would  appear  that  an  extra-cockpit  visual 
display  is  an  effective  way  to  present  visual  information  used  in  some  operational  tasks, 
such  as  landing  on  a carrier,  taxiing,  refueling,  delivering  certain  kinds  of  weapons,  and  air- 
to-air  combat.  In  some  instances,  it  may  be  the  only  effective  way.  In  others,  it  may  be 
effective  but  inefficient,  particularly  when  cost  is  .taken  into  consideration. 

Motion  Fidelity.  Not  much  more  is  known  about  the  influence  of  motion  upon  simu- 
lator training  effectiveness  than  about  visual  displays.  Although  motion  simulation  has 
represented  a significant  portion  of  the  cost  of  simulator  procurement  and  operation  for 
a number  of  years,  the  investigation  of  the  influence  of  motion  upon  simulator  training 
effectiveness  has  been  largely  ignored.  The  first  significant  study  involving  simulator 
motion  in  the  transfer  of  pilot  training  was  reported  in  1975  by  Jacobs  and  Roscoe.  (25) 

The  results  of  the  Jacobs  and  Roscoe  study  provide  evidence  that  transfer  may  not  berxfit 
from  the  presence  of  normal  w’ashout  cockpit  motion.  In  that  study,  training  received  in 
a two-axis  normal  washout  motion  condition,  compared  with  training  in  the  same  device 
without  motion,  resulted  in  nonsignificant  differences  in  amount  of  transfer  to  the  aircraft 
for  these  two  conditions.  There  was  however,  significant  positive  transfer  for  both  motion 
and  no-motior  conditions.  Simdar  resiiltr  have  been  obtained  in  an  unpublished  U.S.  Air 
Force  undergraduate  pilot  training  study  involving  a more  sophisticated  six-ro>:is  motion 
system.  (23) 

The  Jacobs  and  Roscoe  finding  (that,  at  least  with  beginning  trainees,  the  presence 
of  motion  may  not  increase  simulator  training  effectiveness)  must  be  treated  with  caution 
until  investigated  further,  since  there  are  other  studies  suggesting  that,  at  least  under  some 
circumstances,  motic..  may  be  desirable  even  if  not  essential.  For  example,  Fedderson  (26) 
reported  a slight  advantage  in  favor  of  a motion-simulator  trained  group  over  a no-motion 
group  during  brief  transfer  trials  hovering  a helicopter.  More  importantly,  perhaps, 
the  motion  group  in  his  study  reached  asymptotic  performance  in  the  simulator  more 
rapidly,  suggesting  that  simulators  with  motion  may  provide  more  efficient  training,  even 
if  not  more  effective  training.  NASA  researchers  (27)  have  found  that  the  correlation 
between  pilot  performance  in  an  aircraft  and  in  a simulator  increases  with  the  addition  of 
simulator  motion  cues,  where  such  cues  help  the  pilot  in  coping  with  a highly  damped 
or  unstable  vehicle  or  a sluggish  control  system  or,  under  some  circumstances,  where  the 
control  system  is  too  sensitive.  Where  the  aircraft  is  easy  to  fly,  however,  as  is  the  case 
with  the  aircraft  used  in  the  Jacobs  and  Roscoe  study  (Piper  Cherokee)  and  in  the  Air  ^ 

Force  study  (T-37),  motion  has  no  effect.  In  another  NASA  study  (28)  of  the  effects 
of  simulator  motion  on  pilots’  performance  of  a tracking  task,  the  results  from  a moving 
flight  .simulator  resembled  the  results  from  flight  much  more  than  did  those  from  a s 
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motionless  simulator.  Huddleston  and  Rolfe  (29)  reported  that,  using  simulators  without 
motion,  experienced  pilots  are  often  able  to  achieve  acceptable  levels  of  performance,  but 
their  patterns  of  control  response  shov/  that  their  performance  is  achieved  using  a strategy 
different  from  that  used  in  a dynamic  training  environment.  Since  control  strategies 
may  be  important  during  inflight  emergency  maneuvers  where  transfer  of  training  research 
is  not  feasible,  it  would  appear  inadvisable  to  eliminate  motion  from  all  simulators  until 
further  investigation  shows  the  generality  of  the  Jacobs  and  Roscoe  findings.  At  the 
present  time,  we  cannot  be  certain  of  the  role  of  motion  in  simulator  training  effective- 
ness and  efficiency. 

Handling  Characteristics.  Simulators  built  with  the  technology  available  two  decades 
ago  tended  to  have  handling  characteristics  which  were  sometimes  quite  unlike  those  of  an 
aircraft,  and  their  effectiveness  was  limited  largely  by  the  fact  that  pilots  resisted  training 
in  them  or  would  use  them  only  as  procedures  trainers.  (5)  There  were— and  still  are- 
strong  pilot  opinions  that  a simulator  had  to  “feel”  like  an  aircraft  if  it  were  to  be  effec- 
tive. Transfer  studies  of  individual  aircraft  control  parameters,  such  as  a study  of  the 
correspondence  in  stick  pressures  between  a device  and  a training  aircraft  (30),  failed  to 
lend  support  to  the  pilots’  opinions.  Where  the  correspondence  between  the  device  and 
the  aircraft  is  gross,  as  was  found  in  one  device  in  which  forward  pressure  on  the  wheel 
resulted  in  a climb  configuration  (31),  simulator  effectiveness  undoubtedly  will  suffer. 

Thus,  except  for  the  extreme  case  wherein  .simulator  response  characteristics  unlike  those 
of  the  aircraft  can  produce  negative  transfer  of  training,  there  is  little  evidence  that  the 
simulator  must  precisely  duplicate  the  feel  of  the  aircraft  in  order  to  be  effective.  It  is 
possible,  however,  that  even  minor  dissimilarities  in  feel  or  response  could  lead  to  the  same 
kinds  of  potential  problems  found  in  simulators  without  motion,  i.e.,  lower  correlation 
between  simulator  and  flight  performance,  particularly  where  the  more  difficult  to  fly  air- 
craft are  concerned. 

Oxu:  understanding  of  simulator  design  features  in  relation  to  simulator  training  effec- 
tiveness is  quite  limited.  It  is  clear  that  designing  a simulator  is  not  entirely  a matter  of 
duplicating  an  aircraft.  The  physical  correspondence  between  the  simulator  and  the  air- 
craft is  probably  more  related  to  cost,  as  Miller  (32)  indicated  almost  two  decades  ago, 
than  to  training  effectiveness.  If  the  degree  of  correspondence  between  the  device  and 
the  aircraft  is  relevant  to  the  objectives  of  the  intended  training,  training  in  the  simulator 
can  be  made  effective.  Whether  it  is  effective  or  not  is  a matter  related  to  other  factors. 

Training  Problems 

Frequent  note  has  been  taken  of  the  influence  upon  training  effectiveness  of  the 
manner  in  which  a simulator  is  used.  Yet,  the  literature  is  full  of  reports  of  situations  in 
which  the  importance  of  training  program  design  and  execution  seemed  to  be  ignored. 

(See  reference  (5),  for  example.)  Although  there  is  an  increasing  emphasis  upon  effective 
use  of  devices,  current  instances  can  be  cited  of  training  programs  in  which  simulators  are 
misused  or  are  used  inefficiently.  Even  in  simulator  effectiveness  research,  participating 
instructors  often  are  permitted  to  conduct  training  in  various  non-standardized  ways. 

To  list  all  training  program  design  and  execution  variables  which  potentially  influence 
simulator  training  effectiveness  would  be  an  almost  interminable  task.  Any  of  the  numerous 
textbooks  on  human  learning  will  provide  a source  for  identification  of  variables  which 
influence  learning  and  performance,  e.g.,  schedules  of  reinforcement,  meaningfulness  and 
difficulty  of  material  to  be  learned,  size  of  learning  blocks,  and  knowledge  of  results. 
Flexman,  et  al.  (18)  have  shown  how  such  variables  can  be  employed  to  increase  simulator 
and  flight  training  effectiveness. 

The  sequencing  of  simulator  and  aircraft  training  has  been  suggested  as  a factor  which 
could  influence  the  effectiveness  of  simulator  training.  Smode,  et  al.  (2)  concluded  that 
the  evidence  concerning  whether  sequencing  is  influential  was  inconclusive.  Meister, 
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et  al.  (11)  presented  data  which  suggest  that  switching  from  the  aircraft  to  the  simulator 
reduces  performance  in  the  simulator  on  the  following  sessions,  resulting  in  a training 
inefficiency.  While  there  may  be  some  interactive  effects  between  the  sequence,  the  man- 
ner in  which  the  device  is  used,  and  the  design  of  the  device  which  could  influence  effec- 
tiveness, it  would  appear  quite  likely  that  training  in  the  aircraft  before  the  full  benefit 
of  the  simulator  has  been  realized  with  respect  to  a particular  task  would  tend  to  reduce 
the  overall  efficiency  of  the  simulator-device  training  program.  In  an  unpublished  instance 
that  illustrates  this  view,  a fifty-training-hour  program  in  which  the  simulator  was  used 
prior  to  training  in  the  aircraft  became  a sixty-training-hour  program  when  the  sequence 
was  changed  to  mi.x  simulator  and  aircraft  training,  although  other  changes  were  introduced 
concurrently  which  could  have  contributed  in  the  resulting  inefficiency. 

Training  program  content  is  an  obvious  influence  upon  simulator  training  effective- 
ness. A dynamic  flight  simulator  used  only  as  a procedures  trainer,  for  example,  is  not 
being  used  effectively.  It  is  also  believed  that  simulator  training  presented  in  the  context 
of  simulated  mission  activities,  as  opposed  to  abstract  training  exercises,  tends  to  be  more 
effective,  and  the  literature  on  learning  and  forgetting  suggests  that  behavior  learned  within 
.such  a meaningful  context  will  be  less  quickly  forgotten.  (33) 

There  arc  a number  of  other  training  program  factors  which  influence  simulator  train- 
ing efficiency  and  thus  would  lead  to  a higher  TER  value,  although  not  to  increased  effec- 
tiveness per  se.  These  factors  include  the  amount  of  simulator  training,  the  sequence  in 
which  mstruction  is  conducted  in  the  simulator,  the  use  of  individual  (as  opposed  to  group) 
pacing,  training  to  specified  criterion  level?  (as  opposed  to  training  for  fixed  time  periods), 
and  the  extent  to  which  simulator  training  includes  tasks  which  can  be  learned  more  effi- 
ciently in  the  aircraft.  Smode,  et  al.  (2)  pointed  out  a decade  ago  that  little  was  known 
about  how  to  manipulate  such  factors  to  best  advantage.  That  observation  is  still  valid. 

Personnel 

Simulator  training  involves  trainees  and  instructors.  Both  categories  of  personnel 
represent  potential  influences  upon  effectiveness.  The  most  obviously  relevant  considera- 
tions with  respect  to  both  are  their  qualifications  and  prior  experience,  but  occasionally 
other  variables  are  suggested.  For  example,  Meister,  et  al.  (11)  found  a difference  in  the 
effectiveness  of  one  simulator  training  program  for  student  and  operational  pilots  vs. 
-,>'servi.-.ts.  The  difference  could  also  be  attributed  to  considerativ^ns  such  as  fatigue  and 
stress,  factors  which  probably  account  for  many  unexpected  findings  in  transfer  studies. 

The  present  paper  will  discuss  only  the  more  obvious  personnel  factors. 

Trainees.  All  investigations  of  human  learning  are  subject  to  the  influences  of  task- 
related  aptitudes  of  the  learners.  Aptitudes  are  defined  in  terms  of  learning  efficiency, 
and  high-aptitude  students  learn  a given  task  more  rapidly  or  to  a greater  degree  than  do 
low-aptitude  students.  Where  the  training  program  involves  fixed  amounts  of  simulator 
training  time,  high-aptitude  students  learn  more  tasks  co  transfer  to  the  aircraft;  where 
training  is  to  fixed  performance  levels  and  training  time  varies,  high-  and  low-aptitude 
students  achieve  about  equally,  but  high-aptitude  students  require  less  training  time  in 
the  simulator.  A measure  of  simulator  training  efficiency,  such  as  the  TER,  will  yield 
a higher  value  for  high-aptitude  students,  but  this  does  not  indicate  that  the  simulator 
training  program  is  more  effective  with  such  students.  It  is  probably  equally  effective 
with  both  groups  of  students,  but  training  time  in  the  device  wDl  be  shorter  for  one  than 
for  the  other.  Thus,  while  high-aptitude  students  learn  more  efficiently,  aptitude  per  se 
is  not  believed  to  be  an  influence  upon  simulator  training  effectiveness. 

The  influence  upon  simulator  training  effectiveness  of  level  of  trainee  skill  or  amount 
of  prior  flight  experience  is  frequently  questioned.  Many  military  pilots  and  managers 
acknowledge  that  simulators  provide  appropriate  training  for  the  airlines,  where  the 
trainees  are  highly  experienced,  but  insist  that  the  devices  cannot  be  relied  upon  as 


extensively  to  train  less  experienced  military  pilots.  The  skills  possessed  by  these  two 
groups  of  trainees  do  differ,  qualitatively  as  well  as  quantitatively,  and  the  tasks  for 
which  they  undergo  training  are  not  identical.  Therefore,  the  training  they  receive  .should 
not  be  identical  if  it  is  optimally  designed  to  meet  their  respective  training  needs,  and 
the  characteristics  of  the  simulators  involved  in  their  training  should  vary  as  well.  It  doc.s 
not  follow,  however,  that  simulator  training  can  be  appropriately  designed  and  conducted 
for  one  experience  level  trainee  but  not  for  another.  In  fact,  the  experimental  evidence 
does  not  support  the  contention  that  simulator  training  effectiveness  is  influenced  by  level 
of  trainee  experience  in  isolation  from  other  factors.  After  reviewing  a large  number  of 
transfer  of  training  studies,  Micheli  (3)  concluded  that  flight  training  devices  are  effective 
for  both  neophyte  pilot  trainees  and  experienced  airline  pilots. 

Instructors.  After  reviewing  the  literature  on  the  flight  instructor,  Smode,  et  al.  (2) 
concluded  that  experienced  pilots  do  not  make  better  inflight  instructors  than  inexperienced 
pilots.  The  same  conclusion  can  be  extrapolated  to  simulator  instructors.  While  the  evi- 
dence is  .skimpy,  it  appears  that  even  personnel  with  no  flight  experience  can  be  trained 
to  be  effective  simulator  instructors.  For  example,  in  a simulator  training  study  comparing 
an  instructor  with  several  thousand  hours  military  instructor-pilot  experience,  a recent 
flight  training  program  graduate,  and  a non-rated  individual  with  a few  hours  dual  instruc- 
tion but  no  other  aeronautical  experience,  no  significant  differences  were  found  in  the 
in-flight  performance  of  their  students.  (34) 

There  is  some  eviduict  that  not  all  simulator  instructors  ?rs  equally  prepared  for 
their  job.  Hall,  at  al.  (5)  surveyed  a number  of  military  training  programs  and  found  that 
non-rated  enlisted  instructors  were  ill-prepared  as  compared  with  pilots,  particularly  with 
respect  to  relevant  knowledge  of  the  aircraft.  They  also  noted  that  pilots  were  similarly 
ill-prepared  with  respect  to  knowledge  of  the  capabilities  and  limitations  of  the  simulators. 
Since  no  transfer  data  were  reported,  it  cannot  be  determined  whether  this  factor  had  an 
influence  upon  subsequent  in-flight  performance  in  favor  of  either  type  of  instructor. 

Muckier,  et  al.  (14)  observed  that  in  some  cases  a simulator  instructor  mast  provide 
supplementary  information  about  the  in-flight  task  which  might  not  be  available  to  a non- 
rat^  instructor,  thus  presumably  tipping  the  scale  in  favor  of  pilots  as  simulator  instructors. 
Muckier,  et  al.,  also  noted  that  instructor  ability  and  fidelity  of  simulation  are  related  in 
such  fashion  that  as  fidelity  increases,  the  necessary  level  of  instructor  ability  may  decrease, 
and,  conversely,  as  fidelity  decreases,  instructor  ability  must  increase.  This  relationship 
would  tend  to  place  the  more  able  instructor  in  the  lower  fidelity  simulator  where  a 
greater  amount  of  supplementary  information  might  be  required.  It  has  been  my  observa- 
tion that  just  the  opposite  situation  often  obtains.  The  more  experienced  pilots  instruct 
in  high  fidelity  simulators,  while  less  experienced  and  non-rated  personnel  instruct  in 
older,  lower  fidelity  devices. 

Another  consideration  is  whether  there  should  be  one  instructor  or  two  in  a simulator 
training  program.  That  is,  is  simulator  training  effectiveness  influenced  by  whether  the 
simulator  instructor  is  also  the  in-flight  instructor?  While  this  variable  has  not  been  iso- 
lated for  study,  there  appears  to  be  an  increase  in  effectiveness  when  a single  instructor 
is  responsible  for  both  simulator  and  aiicraft  training,  and  It  has  become  a standard  feature 
of  the  simulator  training  programs  developed  by  my  organization.*  One  apparent  benefit 
is  that  the  instruction  given  in  the  simulator  is  more  compatible  with  that  given  in  the  air- 
craft when  only  one  instructor  is  involved,  thus  reducing  any  potential  negative  transfer 
attributable  to  instructor-peculiar  performance  requirements. 

It  often  has  been  assumed  that  the  instructor  is  an  important  factor  influencing 
training  effectiveness,  and  such  may  well  be  the  case.  If  so,  the  influence  must  be 


*The  single  instructor  concept  has  been  used  elsewhere,  at  least  as  early  as  the  late  1940s.  (35) 
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attributable  to  the  manner  in  which  the  instructor  functions,  i.e.,  to  non-standardization 
in  his  administration  of  the  training  program.  There  Ls  insufficient  evidence  available  at 
this  time  to  attribute  the  assumed  influence  to  instructor  experience  or  qualification  per  se, 
assuming  he  has  undergone  an  instructor  training  program  appropriate  to  the  instructional 
task  at  hand. 

Attitudes 

While  the  influence  of  simulator  design  upon  simulator  training  effectiveness  may  not 
always  be  clear,  simulator  design  has  an  impact  upon  instructors  and  trainees,  reflected  in 
their  attitudes,  which  in  turn  has  a great  influence  upon  simulator  training  effectiveness, 
Flexman  described  this  impact  as  follows  (14,  p.  69):  “Fidelity  of  simulation  can  operate 
as  a motivational  variable.  If  the  simulator  looks,  acts,  feels  and  sounds  like  the  airplane, 
then  the  trainee  is  more  likely  to  be  convinced  that  practice  in  the  device  will  be  bene- 
ficial to  him.”  In  circular  fashion,  attitudes  also  influence  simulator  design.  Williges,  et  al. 
(16)  noted  this  phenomenon  when  they  stated  that  decisions  to  include  complex  and 
expensive  motion  systems  in  simulators  are  invariably  determined  by  pilots’  attitudes.  It 
has  been  my  observation  that  fidelity  of  simulation  has  a greater  impact  upon  the  attitude 
of  the  simulator  instructor,  particularly  if  he  is  a pilot,  than  it  has  upon  the  trainee,  and, 
in  turn,  instructor  attitudes  concerning  simulator  training  can  determine  trainee  attitudes. 

The  most  direct  effect  of  trainee  and  instructor  attitude  upon  simulator  training  effec- 
tiveness is  probably  upon  their  willingness  to  engage  in  simulator  training  in  the  first  place. 
That  is,  devices  which  are  viewed  favorably  seem  to  be  used  more  than  those  which  are 
viewed  less  favorably.  If  the  addition  of  a motion  system  or  visual  display  to  a simulator 
will  result  in  favorable  trainee  and  instructor  attitudes  toward  simulator  training  and  hence 
greater  utilization  of  the  device,  it  is  possible  that  more  effective  simulator  training  will 
result  from  the  greater  utilization,  even  though  the  motion  and  visual  per  se  may  contribute 
nothing  directly  to  the  transfer. 

It  would  be  a mistake  to  attribute  all  favorable  attitudes  toward  simulator  training  to 
high  fidelity.  There  are  relatively  low  fidelity  devices  which  are  viewed  favorably  by  many 
trainees  and  instructors,  and  some  quite  sophisticated  devices  have  been  maligned  unbear- 
ably by  some  of  the  same  people.  One  device  used  extensively  by  the  U.S.  Army  as  an 
instrument  trainer  for  a number  of  j*ears  was  extolled  by  the  device  instructors,  maligned 
by  flight  instructors,  and  described  variously  as  “a  beast”  and  “an  aid”  by  trainees.  A 
study  of  the  effectiveness  of  training  conducted  in  the  device  was  less  ambiguous:  it  was 
useless.  (7) 

Except  to  the  extent  that  favorable  attitudes  increase  device  use,  the  effects  of  atti- 
tude upon  simulator  training  appear  to  be  practically  nil.  In  a study  reported  by  Muckier, 
et  al.  (14),  negative  attitudes  toward  a trainer  were  induced  in  an  experimental  group,  by 
stressing  the  device’s  low  fidelity,  while  positive  attitudes  were  induced  in  another  group 
by  stressing  the  same  device’s  training  effectiveness.  During  transfer  trials  in  the  aircraft, 
both  groups  were  found  to  have  benefited,  about  equally,  from  the  device  training,  thus 
indicating  that  the  induced  negative  attitude  did  not  affect  device  training  effectiveness. 

An  interesting  aspect  of  that  study  was  that  the  n^ative  attitude  group  required  more 
training  in  the  device  to  reach  criterion,  so  that  the  TER  value,  had  that  measure  of  effec- 
tiveness been  used,  would  have  been  greater  for  the  positive  attitude  group. 

No  transfer  study  was  found  during  the  current  review  which  indicatoi  that  attitude 
per  se  was  a factor  influencing  simulator  effectiveness.  On  the  basis  of  my  own  experi- 
ence, it  appears  that  just  the  reverse  may  be  the  case:  simulator  training  effectiveness 
influences  attitudes  toward  simulator  training.  I have  observed  abrupt  shifts  in  attitudes, 
particularly  among  instructors  and  training  program  managers,  following  demonstrations 
of  simulator  effectiveness.  In  one  instance,  instructors’  very  negative  attitudes  toward 
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reduced  scale  paper  mockups  of  a cockpit  became  favorable  when  they  discovered  that, 
unknown  to  them,  their  better  students  were  using  these  “devices”  on  their  own.  In 
a study  reported  by  Meyer,  et  al.  (4),  pilots'  opinions  concerning  simulator  training  were 
found  to  be  more  favorable  following  their  participation  in  an  effective  simulator  training 
program  than  were  the  opinions  of  non-participating  pilots. 

In  spite  of  a lack  cf  supporting  research  evidence,  there  Ls  a consensus  among  trainees, 
instructors,  and  administrators  that  favorable  attitudes  toward  simulator  training  increase 
training  effectiveness.  This  probably  is  correct  in  the  sense  that  more  extensive  use  will 
be  made  of  simulators  if  they  are  viewed  favorably.  It  may  be,  however,  that  attitudes 
are  influenced  more  by  simulator  training  effectiveness,  than  the  other  way  around.  A well 
conducted  “test”  of  the  training  effectiveness  of  a simulator  may  be  a very  influential  fac- 
tor in  assuring  that  its  training  value  will  be  realized. 

Expectations 

Many  aviators  accept  the  proposition  that  training  in  a simulator  might  be  helpful, 
but  view  it  as  less  effective  than  training  in  an  aircraft.  It  has  been  my  observation  that 
simulator  training  administered  under  the  control  of  such  individuals  never  exceed  their 
expectations.  If  simulators  are  viewed  as  useful  only  as  procedures  trainers  or  as  instru- 
ment trainers,  they  tend  to  be  used  only  as  procedures  or  instrument  trainers,  even  though 
the  same  devices  might  be  used  more  effectively  by  others  who  view  them  as  offering  a 
greater  range  of  training  opportunities.  If  simulators  are  viewed  as  useful  only  for  the 
initial  stages  of  the  development  of  a particular  skill,  to  be  followed  by  further  develop- 
ment of  that  skill  in  the  aircraft,  simulator  training  is  less  effective  than  if  they  are  viewed 
as  substitutes  for  the  aircraft  to  be  used  for  the  development  of  a particular  skill  to  cri- 
terion before  transferring  to  the  aircraft.  While  simulator  training  may  not  always  prove 
as  effective  as  some  might  expect,  expectations  appear  to  place  a limit  upon  realized 
effectiveness  by  limiting  the  manner  and  extent  of  simulator  training. 

Expectations  can  influence  simulator  training  effectiveness  in  more  subtle  ways  as 
well.  The  expectation  that  a simulator  training  program  virill  prove  ineffective  can  influence 
its  evaluation  in  the  expected  direction.  Research  by  Rosenthal  (36)  has  shown  that, 
even  with  no  intention  to  do  so,  an  experimenter  influences  the  outcome  of  his  research 
in  the  direction  of  his  expectations.  Since  many  “tests”  of  the  effectiveness  of  simulator 
training  are  conducted  by  pilots  who  hold  strong  views  concerning  the  value  of  simulator 
vs.  aircraft  training,  we  must  assume  that  their  expectations  can  and  sometimes  do 
influence  the  test  data.  In  those  instances  in  which  there  is  real  or  perceived  pressure 
from  a higher  authority  to  reach  a particular  finding  concerning  the  utility  of  a particular 
simulator,  the  effect  might  be  even  greater. 

There  is  an  almost  infinite  number  of  factors  which  might  shape  expectations  con- 
cerning simulator  training  effectiveness.  An  obvious  factor  is  prior  experience  with  simu- 
lator training.  The  more  favorable  opinions  of  pilots  toward  simulators  following  participation 
in  an  effective  simulator  training  program  were  noted  above.  Another  factor  may  be  their 
age.  Smode,  et  al.  (2)  noted  that  older  pilots  tend  to  make  poorer  flight  instructors, 
possibly  becaiue  of  a hesitancy  to  adopt  new  teaching  methr^  sudi  as  the  use  of  simula- 
tion. Total  flight  time  is  probably  also  a factor,  since  the  older,  more  experienced  pilots 
are  more  likely  to  have  had  unsatisfactory  experiences  with  old  simulators  and  typically 
put  greater  confidence  in  in-flight  training. 

Increasing  Simulator  Training  Effectiveness 

It  is  unlikely  that  the  effectiveness  of  any  simulator  training  prt^ram  can  be  attributed 
to  a single  influence.  Instead,  all  the  factors  discussed  above,  and  probably  many  more 


30 


r* 


subtle  ones,  act  in  combination  to  produce  effective  simulator  training.  Even  factors 
which  may  not  be  thought  influential  in  isolation  may  serve  as  catalysts.  Effective 
simulator  training  depends  upon  a proper  combination  of  hardware,  program,  personnel, 
and  other  factors. 

Although  progress  has  been  made  over  the  four  decades  since  Edwin  Link  introduced 
his  first  instrument  flight  trainers,  there  is  still  an  element  of  uncertainity  involved  in  the 
design  and  use  of  simulators  in  meeting  training  objectives.  Wheaton  and  Mirabella  (37) 
noted  that  simulator  designers  have  often  been  more  artisans  than  technicians,  and  because 
of  the  informal  nature  of  the  methods  they  use,  it  is  difficult  to  reproduce  i;heir  results 
or  to  train  others  to  produce  effective  devices.  The  same  comment  can  be  applied  to 
training  program  developers,  perhaps  to  an  even  greater  degree.  There  are  artisans  who 
devise  effective  ways  of  using  simulators,  even  apparently  poorly  designed  simulators,  but 
these  artisans  have  not  been  notably  successful  in  training  others  to  produce  effective  sim- 
ulator training  programs.  Conversely,  it  can  be  noted  that  others  have  produced  precious 
little  in  the  way  of  effectiveness,  even  though  working  with  costly  simulators  of  apparently 
excellent  design. 

While  simulator  training  artisans  can  sometimes  produce  spectacular  results,  there  are 
too  few  such  individuals  to  develop  and  continuously  update  all  of  the  simulator  training 
programs  required  by  military  and  civilian  pilot  training  establishments  and  operational 
units.  The  present  paper  was  conceived  as  an  attempt  to  highlight  some  considerations 
which,  if  attended  to,  might  lead  to  increases  in  simulator  training  effectiveness.  It  may 
have  that  effect  in  some  instances,  but  I am  not  convinced  that  our  present  data  base  is 
sufficient  to  that  objective. 

Clearly,  more  research  is  needed  to  increase  our  understanding  of  factors  influencing 
simulator  training  effectiveness.  But  a conceptual  framework  which  could  make  the  con- 
duct of  that  research  more  efficient  and  relevant  to  generalizable  problem  solutions  is 
lacking.  Because  of  this  lack,  training  specialists  have  no  theoretically  acceptable  design 
models  to  follow  and  no  effectiveness  goals  to  seek.  I do  not  see  much  in  the  recent 
research  literature  which  will  provide  the  conceptual  underpinning  for  required  simulator 
training  designs. 

One  problem  is  that  insufficient  information  is  being  disseminated  about  the  design 
of  simulators  and  training  programs,  both  effective  and  ineffective  ones.  Most  published 
simulator  training  r search  reports  state  the  identity  of  the  simulator,  the  experimental 
design  model,  and  the  results.  Often  information  is  not  included  in  the  report  or  other- 
wise available  about  the  simulator’s  design,  the  way  it  was  used,  the  attitudes  and  expecta- 
tions of  the  personnel  involved,  and  other  factors  which  should  be  of  interest  to  someone 
trying  to  apply  the  study’s  results  to  meet  operational  training  requirements.  There  needs 
to  te  much  more  emphasis  upon  how  the  reported  results  were  achieved. 

Attempts  to  apply  research  results  in  the  design  of  simulators  and  training  programs 
ire  important,  but  greater  benefit  can  be  derived  from  study  of  existing  devices  and  pro- 
grams to  locate  features  which  can  be  adopted  in  new  simulator  training  programs  imder 
development.  The  most  useful  model  to  follow  in  the  development  of  effective  simulator 
training  is  that  provided  by  an  existing  application,  modified  to  incorporate  features  from 
other  such  models  as  seems  appropriate. 

The  simulator  training  practitioners  and  researchers  alike  need  more  case  study 
reports  of  simulator  training  applications.  Such  case  studies  would  serve  two  purposes: 
they  would  provide  models  to  be  followed  in  other  applications;  and  they  would  present 
design  data  which  could  then  be  assembled  and  studied  in  efforts  to  develop  conceptual 
models  for  future  applications  and  to  guide  research. 

While  1 do  not  mean  to  relegate  research  to  a lesser  position  of  importance  in  our 
efforts  to  increase  simulator  training  effectiveness,  I feel  that  there  needs  to  be  more 
emphasis  at  the  present  time  upon  gathering  data  about  existing  simulators  and  training 
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programs  so  that  a better  conceptual  framework  can  be  developed  for  such  research.  Our 
theories  need  to  rest  upon  a broader  data  base — data  which  is  derivable  from  present  appli- 
cations. At  the  present  time,  there  needs  to  be  more  u.se  of  the  scientific  method  called 
“naturalistic  observation”  so  that  a broader  data  base  can  be  developed.  Perhaps  the  first 
step  is  to  recognize  a need  for  better  communication  among  practitioners  and  researchers 
about  the  nature  of  effective  simulator  training.  Such  a step  could  lead  to  increased  simu- 
lator training  effectiveness  through  greater  familiarity  with  the  processes  involved  in  simu- 
lator training. 
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